Amazon OpenSearch Serverless has added support for Federal Information Processing Standards (FIPS) compliant endpoints for Data Plane APIs in US East (N. Virginia), US East (Ohio), Canada (Central), AWS GovCloud (US-East), and AWS GovCloud (US-West). The service now meets the security requirements for cryptographic modules as outlined in https://aws.amazon.com/compliance/fips/. Please refer to the https://docs.aws.amazon.com/general/latest/gr/opensearch-service.html#opensearch-service-regions for more information about Amazon OpenSearch Service availability. To learn more about OpenSearch Serverless FIPS, https://docs.aws.amazon.com/opensearch-service/latest/developerguide/fips-compliance-opensearch-serverless.html  http://aws.amazon.com

Amazon OpenSearch Serverless has added support for Federal Information Processing Standards (FIPS) compliant endpoints for Data Plane APIs in US East (N. Virginia), US East (Ohio), Canada (Central), AWS GovCloud (US-East), and AWS GovCloud (US-West). The service now meets the security requirements for cryptographic modules as outlined in Federal Information Processing Standard (FIPS) 140-3. Please refer to the AWS Regional Services List for more information about Amazon OpenSearch Service availability. To learn more about OpenSearch Serverless FIPS, see the documentation.

Amazon OpenSearch Service now supports latest generation Graviton4-based Amazon EC2 instance families. These new instance types are compute optimized (C8g), general purpose (M8g), and memory optimized (R8g, R8gd) instances. AWS Graviton4 processors provide up to 30% better performance than AWS Graviton3 processors with c8g, m8g and r8g & r8gd offering the best price performance for compute-intensive, general purpose, and memory-intensive workloads respectively. To learn more about Graviton4 improvements, please see the on r8g instances and the on c8g & m8g instances. Amazon OpenSearch Service Graviton4 instances are supported on all OpenSearch versions, and Elasticsearch (open source) versions 7.9 and 7.10. One or more than one Graviton4 instance types are now available on Amazon OpenSearch Service across 23 regions globally: US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Asia Pacific (Jakarta), Asia Pacific (Hong Kong), Asia Pacific (Hyderabad), Asia Pacific (Mumbai), Asia Pacific (Malaysia), Asia Pacific (Osaka), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Asia Pacific (Thailand), Canada (Central), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Spain), Europe (Stockholm), South America(Sao Paulo) and AWS GovCloud (US-West). For region specific availability & pricing, visit our . To learn more about Amazon OpenSearch Service and its capabilities, visit our .  

Amazon OpenSearch Service now supports latest generation Graviton4-based Amazon EC2 instance families. These new instance types are compute optimized (C8g), general purpose (M8g), and memory optimized (R8g, R8gd) instances. AWS Graviton4 processors provide up to 30% better performance than AWS Graviton3 processors with c8g, m8g and r8g & r8gd offering the best price performance for compute-intensive, general purpose, and memory-intensive workloads respectively. To learn more about Graviton4 improvements, please see the blog on r8g instances and the blog on c8g & m8g instances. Amazon OpenSearch Service Graviton4 instances are supported on all OpenSearch versions, and Elasticsearch (open source) versions 7.9 and 7.10. One or more than one Graviton4 instance types are now available on Amazon OpenSearch Service across 23 regions globally: US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Asia Pacific (Jakarta), Asia Pacific (Hong Kong), Asia Pacific (Hyderabad), Asia Pacific (Mumbai), Asia Pacific (Malaysia), Asia Pacific (Osaka), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Asia Pacific (Thailand), Canada (Central), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Spain), Europe (Stockholm), South America(Sao Paulo) and AWS GovCloud (US-West). For region specific availability & pricing, visit our pricing page. To learn more about Amazon OpenSearch Service and its capabilities, visit our product page.

You can now perform batch AI inference within Amazon OpenSearch Ingestion pipelines to efficiently enrich and ingest large datasets for Amazon OpenSearch Service domains. Previously, customers used OpenSearch’s AI connectors to Amazon Bedrock, Amazon SageMaker, and 3rd-party services for real-time inference. Inferences generate enrichments such as vector embeddings, predictions, translations, and recommendations to power AI use cases. Real-time inference is ideal for low-latency requirements such as streaming enrichments. Batch inference is ideal for enriching large datasets offline, delivering higher performance and cost efficiency. You can now use the same AI connectors with Amazon OpenSearch Ingestion pipelines as an asynchronous batch inference job to enrich large datasets such as generating and ingesting up to billions of vector embeddings. This feature is available in all regions that support Amazon OpenSearch Ingestion and 2.17+ domains. Learn more from the https://docs.aws.amazon.com/opensearch-service/latest/developerguide/configure-clients-ml-commons-batch.html.

You can now perform batch AI inference within Amazon OpenSearch Ingestion pipelines to efficiently enrich and ingest large datasets for Amazon OpenSearch Service domains. Previously, customers used OpenSearch’s AI connectors to Amazon Bedrock, Amazon SageMaker, and 3rd-party services for real-time inference. Inferences generate enrichments such as vector embeddings, predictions, translations, and recommendations to power AI use cases. Real-time inference is ideal for low-latency requirements such as streaming enrichments. Batch inference is ideal for enriching large datasets offline, delivering higher performance and cost efficiency. You can now use the same AI connectors with Amazon OpenSearch Ingestion pipelines as an asynchronous batch inference job to enrich large datasets such as generating and ingesting up to billions of vector embeddings. This feature is available in all regions that support Amazon OpenSearch Ingestion and 2.17+ domains. Learn more from the documentation.

Amazon CloudWatch and OpenSearch Service integrated analytics experience is now available in 5 additional commercial regions: Asia Pacific (Osaka), Asia Pacific (Seoul), Europe (Milan), Europe (Spain), and US West (N. California). With this integration, CloudWatch Logs customers have two more query languages for log analytics, in addition to CloudWatch Logs Insights QL. Customers can use SQL to analyze data, correlate logs using JOIN, sub-queries, and use SQL functions, namely, JSON, mathematical, datetime, and string functions for intuitive log analytics. They can also use the OpenSearch PPL to filter, aggregate and analyze their data. With a few clicks, CloudWatch Logs customers can create OpenSearch dashboards for VPC, WAF, and CloudTrail logs to monitor, analyze, and troubleshoot using visualizations derived from the logs. OpenSearch customers no longer have to copy logs from CloudWatch for analysis, or create ETL pipelines. Now, they can use OpenSearch Discover to analyze CloudWatch logs in-place, and build indexes and dashboards on CloudWatch Logs. With this launch the integrated experience is now generally available in Asia Pacific (Osaka), Asia Pacific (Seoul), Europe (Milan), Europe (Spain), and US West (N. California) along with regions where https://docs.aws.amazon.com/opensearch-service/latest/developerguide/direct-query-s3.html#direct-query-cloudwatch-logs-regions-table is available. Please read pricing and free tier details on https://aws.amazon.com/cloudwatch/pricing/, and https://aws.amazon.com/opensearch-service/pricing/. To get started, please refer to https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs//CloudWatchLogs-OpenSearch-Dashboards.html and https://docs.aws.amazon.com/opensearch-service/latest/developerguide/direct-query-s3.html.

Amazon CloudWatch and OpenSearch Service integrated analytics experience is now available in 5 additional commercial regions: Asia Pacific (Osaka), Asia Pacific (Seoul), Europe (Milan), Europe (Spain), and US West (N. California). With this integration, CloudWatch Logs customers have two more query languages for log analytics, in addition to CloudWatch Logs Insights QL. Customers can use SQL to analyze data, correlate logs using JOIN, sub-queries, and use SQL functions, namely, JSON, mathematical, datetime, and string functions for intuitive log analytics. They can also use the OpenSearch PPL to filter, aggregate and analyze their data. With a few clicks, CloudWatch Logs customers can create OpenSearch dashboards for VPC, WAF, and CloudTrail logs to monitor, analyze, and troubleshoot using visualizations derived from the logs. OpenSearch customers no longer have to copy logs from CloudWatch for analysis, or create ETL pipelines. Now, they can use OpenSearch Discover to analyze CloudWatch logs in-place, and build indexes and dashboards on CloudWatch Logs. With this launch the integrated experience is now generally available in Asia Pacific (Osaka), Asia Pacific (Seoul), Europe (Milan), Europe (Spain), and US West (N. California) along with regions where OpenSearch Service direct query is available. Please read pricing and free tier details on Amazon CloudWatch Pricing, and OpenSearch Service Pricing. To get started, please refer to Amazon CloudWatch Logs vended dashboard and Amazon OpenSearch Service Developer Guide.

As I was preparing for this week’s roundup, I couldn’t help but reflect on how database technology has evolved over the past decade. It’s fascinating to see how architectural decisions made years ago continue to shape the way we build modern applications. This week brings a special milestone that perfectly captures this evolution in cloud […]

You can now generate AI-powered forecasts and visualizations on time-series data that has been indexed into Amazon OpenSearch domains. Forecasts can be used to enhance various analytics use cases to power insights into trending infrastructure utilization and events, application or business metrics, and more. They can help you anticipate upcoming changes in areas such as business metrics, website traffic, system performance, and more. You can easily get started with this feature by setting up forecasts within https://docs.aws.amazon.com/opensearch-service/latest/developerguide/dashboards.html or the https://docs.aws.amazon.com/opensearch-service/latest/developerguide/application.html. No data science or AI expertise is required. AI-powered forecasts are available in all Amazon OpenSearch Service regions that support OpenSearch 3.1+ domains. Learn more from the https://docs.opensearch.org/docs/latest/observing-your-data/forecast/index/.

Amazon OpenSearch Ingestion now supports cross-account ingestion for push-based sources such as HTTP and OpenTelemetry (OTel). With this launch, customers can easily share OpenSearch Ingestion pipelines across AWS accounts without relying on additional configurations like VPC peering or AWS Transit Gateway. This capability makes it simpler for organizations with multiple accounts to centralize observability and analytics workflows. For example, a central logging team can create ingestion pipelines and grant access to development teams across different accounts, enabling them to ingest logs, metrics, and traces directly into OpenSearch domains or OpenSearch Serverless collections. This reduces operational overhead and lowers the cost of sharing ingestion pipelines across accounts. Cross-account ingestion for Amazon OpenSearch Ingestion is available today in all AWS regions where OpenSearch Ingestion is https://docs.aws.amazon.com/general/latest/gr/opensearch-service.html#opensearch-service-regions. Customers can get started by creating resource policies in the AWS Management Console or using the AWS CLI, and then enabling pipeline endpoints from their VPCs to ingest data seamlessly. To learn more about this feature, see the https://docs.aws.amazon.com/opensearch-service/latest/developerguide/cross-account-pipelines.html and the launch https://aws.amazon.com/blogs/big-data/announcing-cross-account-ingestion-for-amazon-opensearch-service/.

Amazon OpenSearch Ingestion now supports cross-account ingestion for push-based sources such as HTTP and OpenTelemetry (OTel). With this launch, customers can easily share OpenSearch Ingestion pipelines across AWS accounts without relying on additional configurations like VPC peering or AWS Transit Gateway. This capability makes it simpler for organizations with multiple accounts to centralize observability and analytics workflows. For example, a central logging team can create ingestion pipelines and grant access to development teams across different accounts, enabling them to ingest logs, metrics, and traces directly into OpenSearch domains or OpenSearch Serverless collections. This reduces operational overhead and lowers the cost of sharing ingestion pipelines across accounts. Cross-account ingestion for Amazon OpenSearch Ingestion is available today in all AWS regions where OpenSearch Ingestion is offered. Customers can get started by creating resource policies in the AWS Management Console or using the AWS CLI, and then enabling pipeline endpoints from their VPCs to ingest data seamlessly. To learn more about this feature, see the Amazon OpenSearch Service Developer Guide and the launch blog.

We are excited to announce the launch of disk-optimized vector support for Amazon OpenSearch Serverless, offering customers a cost-effective solution for vector search operations without compromising on accuracy and recall rates. This new feature enables organizations to implement high-quality vector search capabilities while significantly reducing operational costs. With the introduction of Disk Optimized Vectors, customers can now choose between memory-optimized and disk-optimized vector storage options. The disk-optimized option delivers the same high accuracy and recall rates as memory-optimized vectors at lower cost. While this option may introduce slightly higher latency, it's ideal for use cases where sub-millisecond response times aren't critical such as semantic search applications, recommendation systems, and other AI-powered search scenarios. Amazon OpenSearch Serverless, our fully managed deployment option, eliminates the complexities of infrastructure management for search and analytics workloads. The service automatically scales compute capacity, measured in OpenSearch Compute Units (OCUs), based on your workload demands. Please refer to the https://docs.aws.amazon.com/general/latest/gr/opensearch-service.html#opensearch-service-regions for more information about Amazon OpenSearch Service availability. To learn more about OpenSearch Serverless, https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless.html

We are excited to announce the launch of disk-optimized vector support for Amazon OpenSearch Serverless, offering customers a cost-effective solution for vector search operations without compromising on accuracy and recall rates. This new feature enables organizations to implement high-quality vector search capabilities while significantly reducing operational costs. With the introduction of Disk Optimized Vectors, customers can now choose between memory-optimized and disk-optimized vector storage options. The disk-optimized option delivers the same high accuracy and recall rates as memory-optimized vectors at lower cost. While this option may introduce slightly higher latency, it's ideal for use cases where sub-millisecond response times aren't critical such as semantic search applications, recommendation systems, and other AI-powered search scenarios. Amazon OpenSearch Serverless, our fully managed deployment option, eliminates the complexities of infrastructure management for search and analytics workloads. The service automatically scales compute capacity, measured in OpenSearch Compute Units (OCUs), based on your workload demands. Please refer to the AWS Regional Services List for more information about Amazon OpenSearch Service availability. To learn more about OpenSearch Serverless, see the documentation.

Amazon OpenSearch Service introduces support for Derived Source, a new feature that can help reduce the amount of storage required for your OpenSearch Service domains. With derived source support, you can skip storing source fields and dynamically derive them when required.  OpenSearch stores each ingested document in the _source field and also indexes individual fields for search. The _source field can consume significant storage space. To reduce storage use, you can configure OpenSearch to skip storing the _source field and instead reconstruct it dynamically when needed, for example, during search, get, mget, reindex, or update operations. Derived Source is available in all regions where OpenSearch 3.1 is supported. The feature is opt-in and can be enabled at index creation using composite index settings. Please refer to the AWS Regional Services List for more information about Amazon OpenSearch Service availability. To learn more about Derived Source, see the https://docs.opensearch.org/latest/field-types/metadata-fields/source/#derived-source.

OpenSearch has introduced Star-Tree Index, a new feature that significantly improves aggregation performance for high-cardinality and multi-dimensional queries. This index pre-aggregates data across configured dimensions and metrics at ingestion time, enabling sub-second response times for frequent aggregations like terms, histogram, and range. Star-Tree Index is designed for real-time analytics and requires no changes to query syntax; OpenSearch automatically uses the optimized path when supported queries are detected. Early benchmarks show faster aggregation performance on large datasets. This makes it ideal for use cases such as observability, personalization, and time-series dashboards. It works best with append-only data and builds during segment refresh/merge, with minimal impact on ingestion throughput. Star-Tree Index is available in all regions where OpenSearch 3.1 is supported. The feature is opt-in and can be enabled at index creation time using composite index settings. Please refer to the AWS Regional Services List for more information about Amazon OpenSearch Service availability. To learn more about Star-Tree Index, see the https://docs.opensearch.org/latest/search-plugins/star-tree-index/

OpenSearch has introduced Star-Tree Index, a new feature that significantly improves aggregation performance for high-cardinality and multi-dimensional queries. This index pre-aggregates data across configured dimensions and metrics at ingestion time, enabling sub-second response times for frequent aggregations like terms, histogram, and range. Star-Tree Index is designed for real-time analytics and requires no changes to query syntax; OpenSearch automatically uses the optimized path when supported queries are detected. Early benchmarks show faster aggregation performance on large datasets. This makes it ideal for use cases such as observability, personalization, and time-series dashboards. It works best with append-only data and builds during segment refresh/merge, with minimal impact on ingestion throughput. Star-Tree Index is available in all regions where OpenSearch 3.1 is supported. The feature is opt-in and can be enabled at index creation time using composite index settings. Please refer to the AWS Regional Services List for more information about Amazon OpenSearch Service availability. To learn more about Star-Tree Index, see the OpenSearch Documentation

Amazon OpenSearch Service introduces support for Derived Source, a new feature that can help reduce the amount of storage required for your OpenSearch Service domains. With derived source support, you can skip storing source fields and dynamically derive them when required.  OpenSearch stores each ingested document in the _source field and also indexes individual fields for search. The _source field can consume significant storage space. To reduce storage use, you can configure OpenSearch to skip storing the _source field and instead reconstruct it dynamically when needed, for example, during search, get, mget, reindex, or update operations. Derived Source is available in all regions where OpenSearch 3.1 is supported. The feature is opt-in and can be enabled at index creation using composite index settings. Please refer to the AWS Regional Services List for more information about Amazon OpenSearch Service availability. To learn more about Derived Source, see the OpenSearch documentation.

You can now run OpenSearch version 3.1 in Amazon OpenSearch Service. OpenSearch 3.1 introduces several improvements in areas like search relevance and performance, and introduces features that simplify development of vector-driven applications for generative AI workloads. This launch incorporates Lucene 10 that enables optimized vector field indexing resulting in faster indexing times and reduced index sizes, sparse indexing for CPU and storage efficiency improvements, and vector quantization to reduce memory usage. Other key areas of improvement include improved range query performance, which benefits log analytics and time-series workloads, and reduced latency for high-cardinality aggregations. This launch also introduces a new https://docs.opensearch.org/latest/search-plugins/search-relevance/using-search-relevance-workbench/, which provides integrated tools for teams to evaluate and optimize search quality through experimentation. Additionally, this launch includes several improvements in vector search capabilities. First, https://opensearch.org/blog/introducing-the-z-score-normalization-technique-for-hybrid-search/ improves hybrid search reliability by reducing the impact of outliers and different score scales. Finally, you can now boost efficiency of searches using https://docs.opensearch.org/latest/vector-search/optimizing-storage/memory-optimized-search/ that enables the Faiss engine to operate efficiently by memory-mapping the index file and using the operating system's file cache to serve search requests. For information on upgrading to OpenSearch 3.1, please see the https://docs.aws.amazon.com/opensearch-service/latest/developerguide/version-migration.html. OpenSearch 3.1 is now available in all AWS Regions where Amazon OpenSearch Service is available.

You can now run OpenSearch version 3.1 in Amazon OpenSearch Service. OpenSearch 3.1 introduces several improvements in areas like search relevance and performance, and introduces features that simplify development of vector-driven applications for generative AI workloads. This launch incorporates Lucene 10 that enables optimized vector field indexing resulting in faster indexing times and reduced index sizes, sparse indexing for CPU and storage efficiency improvements, and vector quantization to reduce memory usage. Other key areas of improvement include improved range query performance, which benefits log analytics and time-series workloads, and reduced latency for high-cardinality aggregations. This launch also introduces a new Search Relevance Workbench, which provides integrated tools for teams to evaluate and optimize search quality through experimentation. Additionally, this launch includes several improvements in vector search capabilities. First, Z-score normalization improves hybrid search reliability by reducing the impact of outliers and different score scales. Finally, you can now boost efficiency of searches using memory-optimized search that enables the Faiss engine to operate efficiently by memory-mapping the index file and using the operating system's file cache to serve search requests. For information on upgrading to OpenSearch 3.1, please see the documentation. OpenSearch 3.1 is now available in all AWS Regions where Amazon OpenSearch Service is available.

As I was preparing for this week’s roundup, I couldn’t help but reflect on how database technology has evolved over the past decade. It’s fascinating to see how architectural decisions made years ago continue to shape the way we build modern applications. This week brings a special milestone that perfectly captures this evolution in cloud […]

You can now generate AI-powered forecasts and visualizations on time-series data that has been indexed into Amazon OpenSearch 3.1+ domains. Forecasts can be used to enhance various analytics use cases to power insights into trending infrastructure utilization and events, application or business metrics, and more. They can help you anticipate upcoming changes in areas such as business metrics, website traffic, system performance, and more. You can easily get started with this feature by setting up forecasts within https://docs.aws.amazon.com/opensearch-service/latest/developerguide/dashboards.html or the https://docs.aws.amazon.com/opensearch-service/latest/developerguide/application.html. No data science or AI expertise is required. AI-powered forecasts are available in all Amazon OpenSearch Service regions that support OpenSearch 3.1 domains. Learn more from the https://docs.opensearch.org/docs/latest/observing-your-data/forecast/index/.

You can now generate AI-powered forecasts and visualizations on time-series data that has been indexed into Amazon OpenSearch 3.1+ domains. Forecasts can be used to enhance various analytics use cases to power insights into trending infrastructure utilization and events, application or business metrics, and more. They can help you anticipate upcoming changes in areas such as business metrics, website traffic, system performance, and more. You can easily get started with this feature by setting up forecasts within OpenSearch dashboards or the OpenSearch UI. No data science or AI expertise is required. AI-powered forecasts are available in all Amazon OpenSearch Service regions that support OpenSearch 3.1 domains. Learn more from the documentation.

Amazon S3 Vectors is a new cloud object store that provides native support for storing and querying vectors at massive scale, offering up to 90% cost reduction compared to conventional approaches while seamlessly integrating with Amazon Bedrock Knowledge Bases, SageMaker, and OpenSearch for AI applications.

Amazon OpenSearch Serverless has added support for Point in Time (PIT) search and SQL in AWS GovCloud US-East and US-West Regions, enabling you to run multiple queries against a dataset fixed at a specific moment. With PIT search you to maintain consistent search results even as your data continues to change, making it particularly useful for applications that require deep pagination or need to preserve a stable view of data across multiple queries. OpenSearch SQL API allows you to leverage your existing SQL skills and tools to analyze data stored in your collections. PIT supports both forward and backward navigation through search results, ensuring consistency even during ongoing data ingestion. This feature is ideal for e-commerce applications, content management systems, and analytics platforms that require reliable and consistent search capabilities across large datasets. SQL and PPL API support addresses the need for familiar query syntax and improved integration with existing analytics tools, benefiting data analysts and developers who work with OpenSearch Serverless collections. Please refer to the AWS Regional Services List for more information about Amazon OpenSearch Service availability. To learn more about OpenSearch Serverless, see the documentation.