How AI Is Built
@howaiisbuilt.fm
Reposted by How AI Is Built
Thanks to @nicolay.fyi for giving me the opportunity to talk about @trustgraph.bsky.social on How AI Is Built!
Labels such as "facts", "observations", and "assertions" take on new meanings when we begin to consider time. Click 👇 to watch the full episode! 🎙️
youtu.be/VpFVAE3L1nk?
Labels such as "facts", "observations", and "assertions" take on new meanings when we begin to consider time. Click 👇 to watch the full episode! 🎙️
youtu.be/VpFVAE3L1nk?
Temporal RAG: Embracing Time for Smarter, Reliable Knowledge Graphs
YouTube video by How AI Is Built
youtu.be
February 17, 2025 at 6:36 PM
Thanks to @nicolay.fyi for giving me the opportunity to talk about @trustgraph.bsky.social on How AI Is Built!
Labels such as "facts", "observations", and "assertions" take on new meanings when we begin to consider time. Click 👇 to watch the full episode! 🎙️
youtu.be/VpFVAE3L1nk?
Labels such as "facts", "observations", and "assertions" take on new meanings when we begin to consider time. Click 👇 to watch the full episode! 🎙️
youtu.be/VpFVAE3L1nk?
Reposted by How AI Is Built
Dropping some new episodes on @howaiisbuilt.fm . Links below.
January 31, 2025 at 12:44 PM
Dropping some new episodes on @howaiisbuilt.fm . Links below.
Reposted by How AI Is Built
Trey and I talk about the different techniques for AI-powered search and how we can combine them to build modern search systems.
Spotify: open.spotify.com/episode/1udV...
Apple: podcasts.apple.com/us/podcast/a...
Spotify: open.spotify.com/episode/1udV...
Apple: podcasts.apple.com/us/podcast/a...
AI-Powered Search: Context Is King, But Your RAG System Ignores Two-Thirds of It | S2 E21
How AI Is Built · Episode
open.spotify.com
January 9, 2025 at 1:58 PM
Trey and I talk about the different techniques for AI-powered search and how we can combine them to build modern search systems.
Spotify: open.spotify.com/episode/1udV...
Apple: podcasts.apple.com/us/podcast/a...
Spotify: open.spotify.com/episode/1udV...
Apple: podcasts.apple.com/us/podcast/a...
Reposted by How AI Is Built
You want the exact opposite.
You want layers of tools aligned in a graph that you can tune, debug, and update in isolation.
Today on How AI Is Built, we are talking to one of the OGs of search: Trey Grainger, the author of AI Powered Search.
www.youtube.com/watch?v=6IQq...
You want layers of tools aligned in a graph that you can tune, debug, and update in isolation.
Today on How AI Is Built, we are talking to one of the OGs of search: Trey Grainger, the author of AI Powered Search.
www.youtube.com/watch?v=6IQq...
AI-Powered Search: Context Is King, But Your RAG System Ignores Two-Thirds of It | S2 E21
YouTube video by How AI Is Built
www.youtube.com
January 9, 2025 at 1:58 PM
You want the exact opposite.
You want layers of tools aligned in a graph that you can tune, debug, and update in isolation.
Today on How AI Is Built, we are talking to one of the OGs of search: Trey Grainger, the author of AI Powered Search.
www.youtube.com/watch?v=6IQq...
You want layers of tools aligned in a graph that you can tune, debug, and update in isolation.
Today on How AI Is Built, we are talking to one of the OGs of search: Trey Grainger, the author of AI Powered Search.
www.youtube.com/watch?v=6IQq...
New episode is out.
The three contexts of search, layered architectures and much more!
The three contexts of search, layered architectures and much more!
Developers treat search as a blackbox.
Throw everything in a vector database and hope something good comes out.
Throw all ranking signals into one big ML model and hope it makes something good out of it.
You don’t want to create this witch’s cauldron.
New episode on @howaiisbuilt.fm
Throw everything in a vector database and hope something good comes out.
Throw all ranking signals into one big ML model and hope it makes something good out of it.
You don’t want to create this witch’s cauldron.
New episode on @howaiisbuilt.fm
January 9, 2025 at 1:59 PM
New episode is out.
The three contexts of search, layered architectures and much more!
The three contexts of search, layered architectures and much more!
Reposted by How AI Is Built
Youtube: www.youtube.com/watch?v=trG5...
Spotify: open.spotify.com/episode/6eyT...
Apple: podcasts.apple.com/us/podcast/c...
Spotify: open.spotify.com/episode/6eyT...
Apple: podcasts.apple.com/us/podcast/c...
Chunking for RAG: Stop Breaking Your Documents Into Meaningless Pieces | S2 E20
YouTube video by How AI Is Built
www.youtube.com
January 3, 2025 at 11:28 AM
Youtube: www.youtube.com/watch?v=trG5...
Spotify: open.spotify.com/episode/6eyT...
Apple: podcasts.apple.com/us/podcast/c...
Spotify: open.spotify.com/episode/6eyT...
Apple: podcasts.apple.com/us/podcast/c...
Reposted by How AI Is Built
The biggest lie in RAG is that semantic search is simple.
The reality is that it's easy to build, it's easy to get up and running, but it's really hard to get right.
And if you don't have a good setup, it's near impossible to debug.
One of the reasons it's really hard is chunking.
The reality is that it's easy to build, it's easy to get up and running, but it's really hard to get right.
And if you don't have a good setup, it's near impossible to debug.
One of the reasons it's really hard is chunking.
January 3, 2025 at 11:28 AM
The biggest lie in RAG is that semantic search is simple.
The reality is that it's easy to build, it's easy to get up and running, but it's really hard to get right.
And if you don't have a good setup, it's near impossible to debug.
One of the reasons it's really hard is chunking.
The reality is that it's easy to build, it's easy to get up and running, but it's really hard to get right.
And if you don't have a good setup, it's near impossible to debug.
One of the reasons it's really hard is chunking.
Reposted by How AI Is Built
Most companies can't afford huge teams labeling AI data.
So, use an AI model to train an AI model.
The big labs like Cohere and OpenAI already use “synthetic data” - AI-generated data that mimics real-world patterns.
The LLMs you use are already trained with it.
youtu.be/thqgKG5lZ8Q
So, use an AI model to train an AI model.
The big labs like Cohere and OpenAI already use “synthetic data” - AI-generated data that mimics real-world patterns.
The LLMs you use are already trained with it.
youtu.be/thqgKG5lZ8Q
How AI Can Start Teaching Itself
YouTube video by How AI Is Built
youtu.be
December 19, 2024 at 12:43 PM
Most companies can't afford huge teams labeling AI data.
So, use an AI model to train an AI model.
The big labs like Cohere and OpenAI already use “synthetic data” - AI-generated data that mimics real-world patterns.
The LLMs you use are already trained with it.
youtu.be/thqgKG5lZ8Q
So, use an AI model to train an AI model.
The big labs like Cohere and OpenAI already use “synthetic data” - AI-generated data that mimics real-world patterns.
The LLMs you use are already trained with it.
youtu.be/thqgKG5lZ8Q
Reposted by How AI Is Built
Youtube: youtu.be/Z9Z820HadIA
Spotify: open.spotify.com/episode/3LAJ...
Apple: podcasts.apple.com/us/podcast/a...
Spotify: open.spotify.com/episode/3LAJ...
Apple: podcasts.apple.com/us/podcast/a...
A Search System That Learns As You Use It (Agentic RAG)
YouTube video by How AI Is Built
youtu.be
December 14, 2024 at 2:07 PM
Youtube: youtu.be/Z9Z820HadIA
Spotify: open.spotify.com/episode/3LAJ...
Apple: podcasts.apple.com/us/podcast/a...
Spotify: open.spotify.com/episode/3LAJ...
Apple: podcasts.apple.com/us/podcast/a...
Reposted by How AI Is Built
Want to learn more? Today on @howaiisbuilt.fm, we are talking to Stephen Batifol from Zilliz. Stephen and I discuss agentic RAG and the future of search - where systems decide their own path to find answers.
What's your take on agentic RAG?
youtu.be/Z9Z820HadIA
What's your take on agentic RAG?
youtu.be/Z9Z820HadIA
A Search System That Learns As You Use It (Agentic RAG)
YouTube video by How AI Is Built
youtu.be
December 14, 2024 at 2:07 PM
Want to learn more? Today on @howaiisbuilt.fm, we are talking to Stephen Batifol from Zilliz. Stephen and I discuss agentic RAG and the future of search - where systems decide their own path to find answers.
What's your take on agentic RAG?
youtu.be/Z9Z820HadIA
What's your take on agentic RAG?
youtu.be/Z9Z820HadIA
Reposted by How AI Is Built
"Instead of being a one-way pipeline, agentic RAG allows you to check, 'Am I actually answering the user's question?'"
Different questions need different approaches.
➡️ 𝗤𝘂𝗲𝗿𝘆-𝗕𝗮𝘀𝗲𝗱 𝗙𝗹𝗲𝘅𝗶𝗯𝗶𝗹𝗶𝘁𝘆:
- Structured data? Use SQL
- Context-rich query? Use vector search
- Date-specific? Apply filters first
Different questions need different approaches.
➡️ 𝗤𝘂𝗲𝗿𝘆-𝗕𝗮𝘀𝗲𝗱 𝗙𝗹𝗲𝘅𝗶𝗯𝗶𝗹𝗶𝘁𝘆:
- Structured data? Use SQL
- Context-rich query? Use vector search
- Date-specific? Apply filters first
December 14, 2024 at 2:07 PM
"Instead of being a one-way pipeline, agentic RAG allows you to check, 'Am I actually answering the user's question?'"
Different questions need different approaches.
➡️ 𝗤𝘂𝗲𝗿𝘆-𝗕𝗮𝘀𝗲𝗱 𝗙𝗹𝗲𝘅𝗶𝗯𝗶𝗹𝗶𝘁𝘆:
- Structured data? Use SQL
- Context-rich query? Use vector search
- Date-specific? Apply filters first
Different questions need different approaches.
➡️ 𝗤𝘂𝗲𝗿𝘆-𝗕𝗮𝘀𝗲𝗱 𝗙𝗹𝗲𝘅𝗶𝗯𝗶𝗹𝗶𝘁𝘆:
- Structured data? Use SQL
- Context-rich query? Use vector search
- Date-specific? Apply filters first
Reposted by How AI Is Built
We talk about how they enable BM25 on PostgreSQL, how they integrate into the Postgres Query engines, and how you can build search in your database.
open.spotify.com/episode/4CXX...
open.spotify.com/episode/4CXX...
Rethinking Search Inside Postgres, From Lexemes to BM25
How AI Is Built · Episode
open.spotify.com
December 5, 2024 at 1:38 PM
We talk about how they enable BM25 on PostgreSQL, how they integrate into the Postgres Query engines, and how you can build search in your database.
open.spotify.com/episode/4CXX...
open.spotify.com/episode/4CXX...
Reposted by How AI Is Built
Not anymore.
ParadeDB is building an open-source PostgreSQL extension to enable search within your database.
Today on How AI Is Built, I am talking to @philippemnoel.bsky.social , the founder and CEO of @paradedb.bsky.social.
youtu.be/RPjGuOcrTsQ
ParadeDB is building an open-source PostgreSQL extension to enable search within your database.
Today on How AI Is Built, I am talking to @philippemnoel.bsky.social , the founder and CEO of @paradedb.bsky.social.
youtu.be/RPjGuOcrTsQ
Rethinking Search Inside Postgres, From Lexemes to BM25
YouTube video by How AI Is Built
youtu.be
December 5, 2024 at 1:38 PM
Not anymore.
ParadeDB is building an open-source PostgreSQL extension to enable search within your database.
Today on How AI Is Built, I am talking to @philippemnoel.bsky.social , the founder and CEO of @paradedb.bsky.social.
youtu.be/RPjGuOcrTsQ
ParadeDB is building an open-source PostgreSQL extension to enable search within your database.
Today on How AI Is Built, I am talking to @philippemnoel.bsky.social , the founder and CEO of @paradedb.bsky.social.
youtu.be/RPjGuOcrTsQ
Reposted by How AI Is Built
Many companies use ElasticSearch or OpenSearch and use 10% of the capacity.
On top, they have to build ETL pipelines.
Get data normalized.
Worry about race conditions.
All in all, when you want to do search on top of your existing database, you are forced to build distributed systems.
#ai
On top, they have to build ETL pipelines.
Get data normalized.
Worry about race conditions.
All in all, when you want to do search on top of your existing database, you are forced to build distributed systems.
#ai
December 5, 2024 at 1:38 PM
Many companies use ElasticSearch or OpenSearch and use 10% of the capacity.
On top, they have to build ETL pipelines.
Get data normalized.
Worry about race conditions.
All in all, when you want to do search on top of your existing database, you are forced to build distributed systems.
#ai
On top, they have to build ETL pipelines.
Get data normalized.
Worry about race conditions.
All in all, when you want to do search on top of your existing database, you are forced to build distributed systems.
#ai
Documentation quality is the silent killer of RAG systems. A single ambiguous sentence might corrupt an entire set of responses. But the hardest part isn't fixing errors - it's finding them.
Check out the episode with Max.
Links to Spotify, Apple in the thread.
Check out the episode with Max.
Links to Spotify, Apple in the thread.
We talk about identifying ambiguities, fixing errors, creating improvement loops in the documents and a lot more.
You can find the episode down below!
♻️ Repost this if you know someone struggling with RAG ♻️
Youtube: youtu.be/RtJY6sIQqcY
You can find the episode down below!
♻️ Repost this if you know someone struggling with RAG ♻️
Youtube: youtu.be/RtJY6sIQqcY
From Ambiguous to AI-Ready: Improving Documentation Quality for RAG Systems | S2 E15
YouTube video by How AI Is Built
youtu.be
November 21, 2024 at 12:07 PM
Documentation quality is the silent killer of RAG systems. A single ambiguous sentence might corrupt an entire set of responses. But the hardest part isn't fixing errors - it's finding them.
Check out the episode with Max.
Links to Spotify, Apple in the thread.
Check out the episode with Max.
Links to Spotify, Apple in the thread.
Reposted by How AI Is Built
LLMs hallucinate.
We want to put the blame on them.
But often it’s our fault.
Many knowledge bases have:
→ Temporal Inconsistencies
- Multiple versions from different time periods
- Historical information without timeline context
>>
We want to put the blame on them.
But often it’s our fault.
Many knowledge bases have:
→ Temporal Inconsistencies
- Multiple versions from different time periods
- Historical information without timeline context
>>
November 21, 2024 at 12:05 PM
LLMs hallucinate.
We want to put the blame on them.
But often it’s our fault.
Many knowledge bases have:
→ Temporal Inconsistencies
- Multiple versions from different time periods
- Historical information without timeline context
>>
We want to put the blame on them.
But often it’s our fault.
Many knowledge bases have:
→ Temporal Inconsistencies
- Multiple versions from different time periods
- Historical information without timeline context
>>
Reposted by How AI Is Built
With RAG these issues are amplified.
We do not look at full documents anymore, but at bits and pieces.
So we have to be extra careful.
Today on @howaiisbuilt.fm we talk to Max Buckley.
Max works at Google and has built a lot of interesting stuff with LLMs to improve knowledge bases for RAG.
>>
We do not look at full documents anymore, but at bits and pieces.
So we have to be extra careful.
Today on @howaiisbuilt.fm we talk to Max Buckley.
Max works at Google and has built a lot of interesting stuff with LLMs to improve knowledge bases for RAG.
>>
November 21, 2024 at 12:05 PM
With RAG these issues are amplified.
We do not look at full documents anymore, but at bits and pieces.
So we have to be extra careful.
Today on @howaiisbuilt.fm we talk to Max Buckley.
Max works at Google and has built a lot of interesting stuff with LLMs to improve knowledge bases for RAG.
>>
We do not look at full documents anymore, but at bits and pieces.
So we have to be extra careful.
Today on @howaiisbuilt.fm we talk to Max Buckley.
Max works at Google and has built a lot of interesting stuff with LLMs to improve knowledge bases for RAG.
>>
Reposted by How AI Is Built
Some query types might not work at all.
It is very costly in terms of storage and compute. We have to keep our indexes in memory to achieve a low enough latency for search.
What we are talking about today works for everything, works out of domain, and is one of the most efficient.
>>
It is very costly in terms of storage and compute. We have to keep our indexes in memory to achieve a low enough latency for search.
What we are talking about today works for everything, works out of domain, and is one of the most efficient.
>>
November 15, 2024 at 12:21 PM
Some query types might not work at all.
It is very costly in terms of storage and compute. We have to keep our indexes in memory to achieve a low enough latency for search.
What we are talking about today works for everything, works out of domain, and is one of the most efficient.
>>
It is very costly in terms of storage and compute. We have to keep our indexes in memory to achieve a low enough latency for search.
What we are talking about today works for everything, works out of domain, and is one of the most efficient.
>>
Reposted by How AI Is Built
People implementing RAG jump straight into vector search.
But vector search has a lot of downsides.
Vector search is not robust out of domain.
Different types of queries need different embedding models with different vector indices.
>>
But vector search has a lot of downsides.
Vector search is not robust out of domain.
Different types of queries need different embedding models with different vector indices.
>>
November 15, 2024 at 12:21 PM
People implementing RAG jump straight into vector search.
But vector search has a lot of downsides.
Vector search is not robust out of domain.
Different types of queries need different embedding models with different vector indices.
>>
But vector search has a lot of downsides.
Vector search is not robust out of domain.
Different types of queries need different embedding models with different vector indices.
>>
Reposted by How AI Is Built
You probably guessed it, we are talking about the OG ranking function in search: BM25.
Today we are back continuing our series on search on @howaiisbuilt.fm with @taidesu.bsky.social.
We talk about BM25, how it works, what makes it great and how you can tailor it to your use-case.
Today we are back continuing our series on search on @howaiisbuilt.fm with @taidesu.bsky.social.
We talk about BM25, how it works, what makes it great and how you can tailor it to your use-case.
November 15, 2024 at 12:21 PM
You probably guessed it, we are talking about the OG ranking function in search: BM25.
Today we are back continuing our series on search on @howaiisbuilt.fm with @taidesu.bsky.social.
We talk about BM25, how it works, what makes it great and how you can tailor it to your use-case.
Today we are back continuing our series on search on @howaiisbuilt.fm with @taidesu.bsky.social.
We talk about BM25, how it works, what makes it great and how you can tailor it to your use-case.
Reposted by How AI Is Built
"Sadly, it's a bit off a snake oil. These long context embedding models have tested basically all of them, not really working well. So it's [best length of chunks] something between like 500 and 1,000 tokens."
Text embeddings are far from perfect.
They struggle with long documents.
>>
Text embeddings are far from perfect.
They struggle with long documents.
>>
November 9, 2024 at 4:52 PM
"Sadly, it's a bit off a snake oil. These long context embedding models have tested basically all of them, not really working well. So it's [best length of chunks] something between like 500 and 1,000 tokens."
Text embeddings are far from perfect.
They struggle with long documents.
>>
Text embeddings are far from perfect.
They struggle with long documents.
>>
Reposted by How AI Is Built
Vector Databases come with their own set of challenges.
The data is too large to be stored on a single node.
We often need to handle 10k to 50k QPS.
Indexes are very slow to build, but we still want to search the fresh data.
>>
The data is too large to be stored on a single node.
We often need to handle 10k to 50k QPS.
Indexes are very slow to build, but we still want to search the fresh data.
>>
November 7, 2024 at 1:37 PM
Vector Databases come with their own set of challenges.
The data is too large to be stored on a single node.
We often need to handle 10k to 50k QPS.
Indexes are very slow to build, but we still want to search the fresh data.
>>
The data is too large to be stored on a single node.
We often need to handle 10k to 50k QPS.
Indexes are very slow to build, but we still want to search the fresh data.
>>
Catch the episode on:
- Youtube: youtu.be/3PEARAf7HEc (now in 4K :D)
- Spotify: open.spotify.com/episode/5lCl...
- Apple: podcasts.apple.com/us/podcast/v...
November 7, 2024 at 1:39 PM
Catch the episode on:
- Youtube: youtu.be/3PEARAf7HEc (now in 4K :D)
- Spotify: open.spotify.com/episode/5lCl...
- Apple: podcasts.apple.com/us/podcast/v...
Reposted by How AI Is Built
“There is no free lunch.”
Every performance optimization comes with tradeoffs in either functionality, flexibility, or cost.
When building search systems, there's a seductive idea that we can optimize everything: fast results, high relevancy, and low costs.
But that’s not the reality.
Every performance optimization comes with tradeoffs in either functionality, flexibility, or cost.
When building search systems, there's a seductive idea that we can optimize everything: fast results, high relevancy, and low costs.
But that’s not the reality.
November 5, 2024 at 5:22 PM
“There is no free lunch.”
Every performance optimization comes with tradeoffs in either functionality, flexibility, or cost.
When building search systems, there's a seductive idea that we can optimize everything: fast results, high relevancy, and low costs.
But that’s not the reality.
Every performance optimization comes with tradeoffs in either functionality, flexibility, or cost.
When building search systems, there's a seductive idea that we can optimize everything: fast results, high relevancy, and low costs.
But that’s not the reality.