Sandro Volpicella
@sandrovolpicella.com
I teach developers about the cloud ⛅
👨🏽💻 Platform Lead @hashnode
📕 New Book: https://cloudwatchbook.com
⛅ AWS Fundamentals Book: https://awsfundamentals.com
😼 Builds https://kitlytics.com
🤗 AWS Community Builder
👨🏽💻 Platform Lead @hashnode
📕 New Book: https://cloudwatchbook.com
⛅ AWS Fundamentals Book: https://awsfundamentals.com
😼 Builds https://kitlytics.com
🤗 AWS Community Builder
Thank you to Honeycomb for inviting me to explore their solutions and sponsoring this content.
🔗 Check out the blog post (includes real-world examples): fandf.co/49E96x0
🔗 Check out the blog post (includes real-world examples): fandf.co/49E96x0
It’s The End Of Observability As We Know It (And I Feel Fine)
In a really broad sense, the history of observability tools over the past couple of decades have been about a pretty simple concept: how do we make terabytes of heterogeneous telemetry data comprehensible to human beings? We’ve seen different companies tackle this in different ways for technology like Ruby on Rails, AWS, Kubernetes, and now OpenTelemetry .
fandf.co
November 26, 2025 at 5:03 PM
Thank you to Honeycomb for inviting me to explore their solutions and sponsoring this content.
🔗 Check out the blog post (includes real-world examples): fandf.co/49E96x0
🔗 Check out the blog post (includes real-world examples): fandf.co/49E96x0
I've always loved Honeycomb's resources on observability 📚
Before writing my CloudWatch book, I read their "Observability Engineering" book. It's amazing.
Their blog is also a gold mine for understanding that observability is more than just dashboards and three pillars 😉
Before writing my CloudWatch book, I read their "Observability Engineering" book. It's amazing.
Their blog is also a gold mine for understanding that observability is more than just dashboards and three pillars 😉
November 26, 2025 at 5:03 PM
I've always loved Honeycomb's resources on observability 📚
Before writing my CloudWatch book, I read their "Observability Engineering" book. It's amazing.
Their blog is also a gold mine for understanding that observability is more than just dashboards and three pillars 😉
Before writing my CloudWatch book, I read their "Observability Engineering" book. It's amazing.
Their blog is also a gold mine for understanding that observability is more than just dashboards and three pillars 😉
But the actual root cause is something completely else.
I think AI shouldn't be 100% only text-based.
It should also show you the exact investigation path in the tools you already know how to use.
You need to use it in a combination. If it helps you get to the goals faster, that's great!
I think AI shouldn't be 100% only text-based.
It should also show you the exact investigation path in the tools you already know how to use.
You need to use it in a combination. If it helps you get to the goals faster, that's great!
November 26, 2025 at 5:03 PM
But the actual root cause is something completely else.
I think AI shouldn't be 100% only text-based.
It should also show you the exact investigation path in the tools you already know how to use.
You need to use it in a combination. If it helps you get to the goals faster, that's great!
I think AI shouldn't be 100% only text-based.
It should also show you the exact investigation path in the tools you already know how to use.
You need to use it in a combination. If it helps you get to the goals faster, that's great!
> "Account 42783330 has occasional spikes over 3s. Could you use caching? Or I saw two database calls that aren't necessarily needed at this part of the code."
Sounds good at a first glance.
But I want to actually see the root cause.
Maybe caching is just fixing the symptom.
Sounds good at a first glance.
But I want to actually see the root cause.
Maybe caching is just fixing the symptom.
November 26, 2025 at 5:03 PM
> "Account 42783330 has occasional spikes over 3s. Could you use caching? Or I saw two database calls that aren't necessarily needed at this part of the code."
Sounds good at a first glance.
But I want to actually see the root cause.
Maybe caching is just fixing the symptom.
Sounds good at a first glance.
But I want to actually see the root cause.
Maybe caching is just fixing the symptom.
Let's say you have a production issue like your checkout is slow.
As a developer, you don't care about which tool you use to find the issues.
You care about finding the solution.
With proper AI it could look like that:
> "My checkout is slow for some customers, why?"
…🤖
As a developer, you don't care about which tool you use to find the issues.
You care about finding the solution.
With proper AI it could look like that:
> "My checkout is slow for some customers, why?"
…🤖
November 26, 2025 at 5:03 PM
Let's say you have a production issue like your checkout is slow.
As a developer, you don't care about which tool you use to find the issues.
You care about finding the solution.
With proper AI it could look like that:
> "My checkout is slow for some customers, why?"
…🤖
As a developer, you don't care about which tool you use to find the issues.
You care about finding the solution.
With proper AI it could look like that:
> "My checkout is slow for some customers, why?"
…🤖
This is what Austin states: SPEED MATTERS.
And I like that take. In the end, you want to get results fast. You don't care how you get the results.
This is where AI & LLMs come in.
They're just much faster at digging through tons of data.
And I like that take. In the end, you want to get results fast. You don't care how you get the results.
This is where AI & LLMs come in.
They're just much faster at digging through tons of data.
November 26, 2025 at 5:03 PM
This is what Austin states: SPEED MATTERS.
And I like that take. In the end, you want to get results fast. You don't care how you get the results.
This is where AI & LLMs come in.
They're just much faster at digging through tons of data.
And I like that take. In the end, you want to get results fast. You don't care how you get the results.
This is where AI & LLMs come in.
They're just much faster at digging through tons of data.
- Each account runs different services that interact with each other.
To be able to find issues you use tools that observability providers have built for you like:
📜 Logs
📊 Dashboards
🕸️ Traces
Every feature is there for you to help you find the problem FASTER.
To be able to find issues you use tools that observability providers have built for you like:
📜 Logs
📊 Dashboards
🕸️ Traces
Every feature is there for you to help you find the problem FASTER.
November 26, 2025 at 5:03 PM
- Each account runs different services that interact with each other.
To be able to find issues you use tools that observability providers have built for you like:
📜 Logs
📊 Dashboards
🕸️ Traces
Every feature is there for you to help you find the problem FASTER.
To be able to find issues you use tools that observability providers have built for you like:
📜 Logs
📊 Dashboards
🕸️ Traces
Every feature is there for you to help you find the problem FASTER.
Don't forget to change the setting in your API Gateway setting.
And make sure your integration supports it as well 😉
And make sure your integration supports it as well 😉
November 26, 2025 at 3:02 PM
Don't forget to change the setting in your API Gateway setting.
And make sure your integration supports it as well 😉
And make sure your integration supports it as well 😉
Hey, I'm Sandro — full-stack engineer building real-world AWS apps ⛅
I share practical serverless insights with 11,200+ developers at lnkd.in/dAGdBiQZ
I share practical serverless insights with 11,200+ developers at lnkd.in/dAGdBiQZ
LinkedIn
This link will take you to a page that’s not on LinkedIn
www.linkedin.com
November 25, 2025 at 2:58 PM
Hey, I'm Sandro — full-stack engineer building real-world AWS apps ⛅
I share practical serverless insights with 11,200+ developers at lnkd.in/dAGdBiQZ
I share practical serverless insights with 11,200+ developers at lnkd.in/dAGdBiQZ
7️⃣ Test in Console
Use "Retrieval only" mode to see raw results.
Results include source chunks with metadata attached.
The beauty of this setup? ✨
You get semantic search with sources attached.
No infrastructure to manage beyond S3.
Just S3, Bedrock, and you're done.
---
Use "Retrieval only" mode to see raw results.
Results include source chunks with metadata attached.
The beauty of this setup? ✨
You get semantic search with sources attached.
No infrastructure to manage beyond S3.
Just S3, Bedrock, and you're done.
---
November 25, 2025 at 2:58 PM
7️⃣ Test in Console
Use "Retrieval only" mode to see raw results.
Results include source chunks with metadata attached.
The beauty of this setup? ✨
You get semantic search with sources attached.
No infrastructure to manage beyond S3.
Just S3, Bedrock, and you're done.
---
Use "Retrieval only" mode to see raw results.
Results include source chunks with metadata attached.
The beauty of this setup? ✨
You get semantic search with sources attached.
No infrastructure to manage beyond S3.
Just S3, Bedrock, and you're done.
---
5️⃣ Choose S3 Vectors as Vector Store
Your embeddings are stored directly in S3.
No separate vector database to manage! 🚀
6️⃣ Sync Your Data
Click "Start Sync" to embed all your content.
Watch progress in CloudWatch.
First sync takes time (100+ posts took us ~10 minutes) ⏱️
Your embeddings are stored directly in S3.
No separate vector database to manage! 🚀
6️⃣ Sync Your Data
Click "Start Sync" to embed all your content.
Watch progress in CloudWatch.
First sync takes time (100+ posts took us ~10 minutes) ⏱️
November 25, 2025 at 2:58 PM
5️⃣ Choose S3 Vectors as Vector Store
Your embeddings are stored directly in S3.
No separate vector database to manage! 🚀
6️⃣ Sync Your Data
Click "Start Sync" to embed all your content.
Watch progress in CloudWatch.
First sync takes time (100+ posts took us ~10 minutes) ⏱️
Your embeddings are stored directly in S3.
No separate vector database to manage! 🚀
6️⃣ Sync Your Data
Click "Start Sync" to embed all your content.
Watch progress in CloudWatch.
First sync takes time (100+ posts took us ~10 minutes) ⏱️
We went with fixed-size chunking: 2000 tokens with 10% overlap.
Hierarchical chunking would probably work better for markdown with headings.
But fixed-size was simpler to start with.
4️⃣ Select Embedding Model
We picked Titan Text Embeddings V2 🎉
Any activated Bedrock model works here.
Hierarchical chunking would probably work better for markdown with headings.
But fixed-size was simpler to start with.
4️⃣ Select Embedding Model
We picked Titan Text Embeddings V2 🎉
Any activated Bedrock model works here.
November 25, 2025 at 2:58 PM
We went with fixed-size chunking: 2000 tokens with 10% overlap.
Hierarchical chunking would probably work better for markdown with headings.
But fixed-size was simpler to start with.
4️⃣ Select Embedding Model
We picked Titan Text Embeddings V2 🎉
Any activated Bedrock model works here.
Hierarchical chunking would probably work better for markdown with headings.
But fixed-size was simpler to start with.
4️⃣ Select Embedding Model
We picked Titan Text Embeddings V2 🎉
Any activated Bedrock model works here.
Add metadata to each object (service, tags, slug, date).
This metadata becomes filterable later in queries.
2️⃣ Create Knowledge Base
Go to Bedrock Console → Knowledge Bases → Create.
Select S3 as your data source.
3️⃣ Configure Chunking
This metadata becomes filterable later in queries.
2️⃣ Create Knowledge Base
Go to Bedrock Console → Knowledge Bases → Create.
Select S3 as your data source.
3️⃣ Configure Chunking
November 25, 2025 at 2:58 PM
Add metadata to each object (service, tags, slug, date).
This metadata becomes filterable later in queries.
2️⃣ Create Knowledge Base
Go to Bedrock Console → Knowledge Bases → Create.
Select S3 as your data source.
3️⃣ Configure Chunking
This metadata becomes filterable later in queries.
2️⃣ Create Knowledge Base
Go to Bedrock Console → Knowledge Bases → Create.
Select S3 as your data source.
3️⃣ Configure Chunking
First copy is free yes!
After that it isn't free anymore. Not sure if its a downside but something to consider :)
After that it isn't free anymore. Not sure if its a downside but something to consider :)
November 24, 2025 at 4:01 PM
First copy is free yes!
After that it isn't free anymore. Not sure if its a downside but something to consider :)
After that it isn't free anymore. Not sure if its a downside but something to consider :)
No complex setup. Just centralized observability.
November 24, 2025 at 3:00 PM
No complex setup. Just centralized observability.
| 𝚏𝚒𝚕𝚝𝚎𝚛 @𝚊𝚠𝚜.𝚊𝚌𝚌𝚘𝚞𝚗𝚝 = 𝟷𝟸𝟹𝟺𝟻𝟼𝟽𝟾𝟿𝟶𝟷𝟸
| 𝚏𝚒𝚕𝚝𝚎𝚛 @𝚊𝚠𝚜.𝚛𝚎𝚐𝚒𝚘𝚗 = '𝚞𝚜-𝚠𝚎𝚜𝚝-𝟸'
| 𝚜𝚘𝚛𝚝 @𝚝𝚒𝚖𝚎𝚜𝚝𝚊𝚖𝚙 𝚍𝚎𝚜𝚌
━━━━━━━━━━━━━━━━━━━━━━
That's it.
Logs flow automatically from all source accounts to your central logging account.
| 𝚏𝚒𝚕𝚝𝚎𝚛 @𝚊𝚠𝚜.𝚛𝚎𝚐𝚒𝚘𝚗 = '𝚞𝚜-𝚠𝚎𝚜𝚝-𝟸'
| 𝚜𝚘𝚛𝚝 @𝚝𝚒𝚖𝚎𝚜𝚝𝚊𝚖𝚙 𝚍𝚎𝚜𝚌
━━━━━━━━━━━━━━━━━━━━━━
That's it.
Logs flow automatically from all source accounts to your central logging account.
November 24, 2025 at 3:00 PM
| 𝚏𝚒𝚕𝚝𝚎𝚛 @𝚊𝚠𝚜.𝚊𝚌𝚌𝚘𝚞𝚗𝚝 = 𝟷𝟸𝟹𝟺𝟻𝟼𝟽𝟾𝟿𝟶𝟷𝟸
| 𝚏𝚒𝚕𝚝𝚎𝚛 @𝚊𝚠𝚜.𝚛𝚎𝚐𝚒𝚘𝚗 = '𝚞𝚜-𝚠𝚎𝚜𝚝-𝟸'
| 𝚜𝚘𝚛𝚝 @𝚝𝚒𝚖𝚎𝚜𝚝𝚊𝚖𝚙 𝚍𝚎𝚜𝚌
━━━━━━━━━━━━━━━━━━━━━━
That's it.
Logs flow automatically from all source accounts to your central logging account.
| 𝚏𝚒𝚕𝚝𝚎𝚛 @𝚊𝚠𝚜.𝚛𝚎𝚐𝚒𝚘𝚗 = '𝚞𝚜-𝚠𝚎𝚜𝚝-𝟸'
| 𝚜𝚘𝚛𝚝 @𝚝𝚒𝚖𝚎𝚜𝚝𝚊𝚖𝚙 𝚍𝚎𝚜𝚌
━━━━━━━━━━━━━━━━━━━━━━
That's it.
Logs flow automatically from all source accounts to your central logging account.
All logs now include two fields:
• @aws.account - source account
• @aws.region - source region
Use CloudWatch Logs Insights to filter:
𝚏𝚒𝚎𝚕𝚍𝚜 @𝚝𝚒𝚖𝚎𝚜𝚝𝚊𝚖𝚙, @𝚊𝚠𝚜.𝚊𝚌𝚌𝚘𝚞𝚗𝚝, @𝚊𝚠𝚜.𝚛𝚎𝚐𝚒𝚘𝚗, @𝚖𝚎𝚜𝚜𝚊𝚐𝚎
• @aws.account - source account
• @aws.region - source region
Use CloudWatch Logs Insights to filter:
𝚏𝚒𝚎𝚕𝚍𝚜 @𝚝𝚒𝚖𝚎𝚜𝚝𝚊𝚖𝚙, @𝚊𝚠𝚜.𝚊𝚌𝚌𝚘𝚞𝚗𝚝, @𝚊𝚠𝚜.𝚛𝚎𝚐𝚒𝚘𝚗, @𝚖𝚎𝚜𝚜𝚊𝚐𝚎
November 24, 2025 at 3:00 PM
All logs now include two fields:
• @aws.account - source account
• @aws.region - source region
Use CloudWatch Logs Insights to filter:
𝚏𝚒𝚎𝚕𝚍𝚜 @𝚝𝚒𝚖𝚎𝚜𝚝𝚊𝚖𝚙, @𝚊𝚠𝚜.𝚊𝚌𝚌𝚘𝚞𝚗𝚝, @𝚊𝚠𝚜.𝚛𝚎𝚐𝚒𝚘𝚗, @𝚖𝚎𝚜𝚜𝚊𝚐𝚎
• @aws.account - source account
• @aws.region - source region
Use CloudWatch Logs Insights to filter:
𝚏𝚒𝚎𝚕𝚍𝚜 @𝚝𝚒𝚖𝚎𝚜𝚝𝚊𝚖𝚙, @𝚊𝚠𝚜.𝚊𝚌𝚌𝚘𝚞𝚗𝚝, @𝚊𝚠𝚜.𝚛𝚎𝚐𝚒𝚘𝚗, @𝚖𝚎𝚜𝚜𝚊𝚐𝚎
• Source Accounts (account IDs, OUs, or entire org)
• Source Regions
Example:
Create a rule for your Production OU that copies all us-east-1 and eu-west-1 logs to your central account.
━━━━━━━━━━━━━━━━━━━━━━
𝗦𝘁𝗲𝗽 𝟯: 𝗤𝘂𝗲𝗿𝘆 𝗖𝗲𝗻𝘁𝗿𝗮𝗹𝗶𝘇𝗲𝗱 𝗟𝗼𝗴𝘀
• Source Regions
Example:
Create a rule for your Production OU that copies all us-east-1 and eu-west-1 logs to your central account.
━━━━━━━━━━━━━━━━━━━━━━
𝗦𝘁𝗲𝗽 𝟯: 𝗤𝘂𝗲𝗿𝘆 𝗖𝗲𝗻𝘁𝗿𝗮𝗹𝗶𝘇𝗲𝗱 𝗟𝗼𝗴𝘀
November 24, 2025 at 3:00 PM
• Source Accounts (account IDs, OUs, or entire org)
• Source Regions
Example:
Create a rule for your Production OU that copies all us-east-1 and eu-west-1 logs to your central account.
━━━━━━━━━━━━━━━━━━━━━━
𝗦𝘁𝗲𝗽 𝟯: 𝗤𝘂𝗲𝗿𝘆 𝗖𝗲𝗻𝘁𝗿𝗮𝗹𝗶𝘇𝗲𝗱 𝗟𝗼𝗴𝘀
• Source Regions
Example:
Create a rule for your Production OU that copies all us-east-1 and eu-west-1 logs to your central account.
━━━━━━━━━━━━━━━━━━━━━━
𝗦𝘁𝗲𝗽 𝟯: 𝗤𝘂𝗲𝗿𝘆 𝗖𝗲𝗻𝘁𝗿𝗮𝗹𝗶𝘇𝗲𝗱 𝗟𝗼𝗴𝘀
This is a one-time setup and you're done.
━━━━━━━━━━━━━━━━━━━━━━
𝗦𝘁𝗲𝗽 𝟮: 𝗖𝗿𝗲𝗮𝘁𝗲 𝗖𝗲𝗻𝘁𝗿𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝗥𝘂𝗹𝗲𝘀
Log into your logging account.
CloudWatch → Settings → Organization.
You can now create rules!
Up to 50 rules per organization.
Each rule needs:
• Name
━━━━━━━━━━━━━━━━━━━━━━
𝗦𝘁𝗲𝗽 𝟮: 𝗖𝗿𝗲𝗮𝘁𝗲 𝗖𝗲𝗻𝘁𝗿𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝗥𝘂𝗹𝗲𝘀
Log into your logging account.
CloudWatch → Settings → Organization.
You can now create rules!
Up to 50 rules per organization.
Each rule needs:
• Name
November 24, 2025 at 3:00 PM
This is a one-time setup and you're done.
━━━━━━━━━━━━━━━━━━━━━━
𝗦𝘁𝗲𝗽 𝟮: 𝗖𝗿𝗲𝗮𝘁𝗲 𝗖𝗲𝗻𝘁𝗿𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝗥𝘂𝗹𝗲𝘀
Log into your logging account.
CloudWatch → Settings → Organization.
You can now create rules!
Up to 50 rules per organization.
Each rule needs:
• Name
━━━━━━━━━━━━━━━━━━━━━━
𝗦𝘁𝗲𝗽 𝟮: 𝗖𝗿𝗲𝗮𝘁𝗲 𝗖𝗲𝗻𝘁𝗿𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝗥𝘂𝗹𝗲𝘀
Log into your logging account.
CloudWatch → Settings → Organization.
You can now create rules!
Up to 50 rules per organization.
Each rule needs:
• Name