Why Is Enterprise Search So Hard That Even Google Couldn't Crack It? | Bolt Picks

**Glean is an AI-powered enterprise search and knowledge management platform founded in 2019. Starting with internal enterprise search, it has combined LLM capabilities with connections to over 100 SaaS applications, enabling users to not only get personalized search results but also conduct cross-application searches across enterprise data. It is widely regarded as the most advanced company in the enterprise AI search space today.

Glean founder Arvind Jain spent over 11 years at Google as an algorithm engineer. After leaving Google, he founded Rubrik, a cloud data management platform. Recently, Arvind Jain shared Glean's experience building enterprise AI search on the podcast Training Data, with particular attention to what makes the product so difficult. We've compiled and translated portions of the conversation. The original podcast can be accessed via the "read more" link.

Figure | Podcast Shownotes

📝 Summary 1. Data infrastructure: Building effective AI applications requires robust data infrastructure, including deep integration with enterprise systems, security and permissions frameworks, and complex knowledge graphs that understand the relationships between people and content.

2. The importance of AI understanding enterprise content: Enterprise search and AI applications need to understand organizational structure, user roles, and access permissions, constructing contextual understanding that is critical for delivering relevant results.

3. Learning from human behavior: Excellent AI systems learn from how people work, tracking document engagement and interactions in communication tools to improve system relevance and usefulness.

4. Start with clear value: First deliver clear value to users through core functionality like enterprise search, then expand to more AI features. This approach builds trust and provides a data foundation for advanced capabilities.

5. Autonomous AI assistants: The next phase of AI is proactive assistants that can anticipate needs and provide guidance in daily work, similar to executive assistants. Realizing this vision requires excellence in both foundational and advanced aspects of enterprise AI.

Part.01

Technical Challenges

1. Sonya Huang: "Google for Work" — using powerful search capabilities to help solve work problems, find information, or improve efficiency has been every CIO's dream. Yet even Google has failed to crack this domain over the past two decades. Can you briefly explain why this problem is so difficult, and how you approached it?

Arvind Jain: Search is in some sense like "magic." Users can ask any question and expect the system to give them the exact right answer, so expectations are always sky-high. It's a thorny problem, especially in the enterprise. Because enterprise information is scattered across numerous systems, simply knowing where everything lives is already hard; judging accuracy and timeliness is even harder. So building such a system is full of challenges.

Solving it requires massive resources, which in many ways is unfriendly to startups. Before SaaS, you couldn't even build this product. Just connecting all of a company's data to a search system would require working closely with the enterprise for a full year, not to mention actually making that information searchable.

2. Pat Grady: What's interesting about Glean is that it's the first, and so far the most successful, demonstration of what an enterprise AI application should look like. We'll talk about AI plenty later, but I know your tech stack has many layers — infrastructure, knowledge graphs, and so on. Can you start by explaining what you had to do first before you could fully leverage AI capabilities?

Arvind Jain: The first step of search is making data and knowledge searchable. So Glean's first step was deep information integration.

Second, the difference between enterprise search and Google web search is that most information inside companies is inherently private. It's not search if you just throw files into a company information system and let everyone see them. You have to understand who is authorized for every piece of content; when someone searches, the system needs to know who they are and return only what they have permission to see. This is Glean's "management layer." Understanding these permission structures across systems is complex.

Third, most similar products in the past failed to solve this: you need to know which documents go to which people, not just list information as an index. Say a new hire searches for onboarding information — are they on the marketing team or are they an engineer? Which document do they need? So understanding the relationship between people and knowledge matters. Glean built a deep knowledge graph that understands each department's function and their documents. It constructs what information is most relevant when a person searches for a document.

For us, AI was important from day one. From the start, we worked on understanding what a user actually wants when they search, not limiting ourselves to their search keywords — all of this was before combining with large language models.

3. Sonya Huang: I think one of Google's core strengths is optimizing the ranking of search answers, putting the information users most want at the top. In public web search, this benefits from massive web data and link relationships. How do you achieve this at Glean?

Arvind Jain: This is indeed central to the product, and we've invested heavily in building an effective search ranking system. I can give some examples of how we determine which documents should be prioritized for a given question.

First, if a document is frequently viewed within a company, that's clearly a signal it has some importance — employees are willing to spend time reading it. Second, if a document was created in the last week or two and has high engagement, that shows it's being paid attention to and the information isn't stale. Additionally, some documents may not have many viewers across the entire company but are used frequently within a specific team, which likely indicates they're particularly relevant to that team.

Another example: if someone is too lazy to use a search tool and instead asks a question on Slack, and a colleague responds with a document link that the asker then likes — this interaction means that document is a good answer. So when others ask similar questions, the system automatically associates that document, which is extremely useful.

In enterprise environments, collecting these effective signals is completely different from web search. Google is the gateway to knowledge search, but in enterprises, much knowledge acquisition doesn't happen through search at all. So we need to analyze activity data from multiple systems, including communication systems and document management systems, and observe employee behavior patterns. By continuously learning these internal behaviors and gathering more relevant information, we can make the ranking system more precise and effective.

Part.02

Enterprise Search vs. RAG

4. Sonya Huang: I think Glean is the expert at making RAG (Retrieval-Augmented Generation) truly serve enterprise content. Can you explain RAG in simple, accessible terms? What's the secret to how it works? What are some challenges that few people talk about? And what can RAG do that ordinary chat interfaces can't?

Arvind Jain: The well-known models on the market — GPT, Gemini, Claude — these are all trained on globally public knowledge and data. So if you ask ChatGPT something like "How many days of paid leave does my company's policy allow?" it completely can't answer. Because the answer belongs to the company's private information, and the model hasn't been trained on it.

To enable models to handle enterprise private data and achieve truly intelligent answers, we rely on the RAG architecture. RAG works like this: you ask a question, and first use a search engine to find potentially relevant documents or content snippets. Then you send these documents to a model, say GPT, telling it, "This is my question, and this is the enterprise knowledge I think might be relevant — please answer based on this information." Today, most enterprise AI applications are built on this architecture. The only way to combine enterprise private data with the powerful capabilities of language models is through a search engine sitting in the middle. Over the past five years, we've developed a search engine based on enterprise content at Glean. This not only makes us one of the best RAG systems, but also supports our end-to-end user application — Glean Assistant — while helping enterprises build more applications using RAG.

Although this architecture has gradually become the standard approach for building AI applications, I think it's still full of challenges. Using RAG to create excellent AI applications isn't easy. First, while models themselves are powerful, they're still emerging technology — for instance, they hallucinate. And the RAG architecture introduces another layer of complex technology, essentially connecting two imperfect complex systems together. So you'll find RAG-based applications frequently have issues, such as failing to find the correct knowledge snippets during retrieval, or getting outdated information, all of which can lead the model to produce wrong results.

5. Pat Grady: Borrowing your earlier framing, phase one was enterprise search, phase two is the applications platform. How do you see the relationship between enterprise search and RAG? Is it a containing-and-contained relationship? Similar but different concepts? Or exactly the same thing?

Arvind Jain: I think enterprise search and RAG are in some sense one and the same. Their core technology is integrating all enterprise knowledge into a search system so that when users ask questions, the system can return relevant information. That's the technical core.

You can use this technology as a standalone product, like Glean's search product, which lets users directly ask questions and get documents relevant to their questions; or you can use it as an API layer for AI applications, as a component in the overall AI application architecture. So they're similar in that regard.

However, many RAG-based applications in enterprises today use simplified versions of retrieval systems, typically vector search, without complete enterprise content. I think that's the key difference. Our strategy has always been focused on building a standalone search system that's good enough to face users directly as an independent product. That's the real test of search effectiveness.

6. Sonya Huang: Could you say that your technical advances in search result ranking are key?

Arvind Jain: Yes. I think building these systems yourself is extremely challenging — hard to complete in just a few weeks. These systems need to be stable, robust, and truly deliver value inside enterprises, not just be an AI prototype for demonstration. That's a hard problem. Now you can make an impressive AI demo in a day, or even two hours, but building something truly practical that can continuously support enterprise operations is still quite difficult.

Part.03

How Glean Changes How People Work

7. Pat Grady: In your observation, what notable changes have you seen in customers' daily work patterns before and after using Glean? How does it help companies work better?

Arvind Jain: Glean is a widely used product, and we're often surprised by what customers use Glean for. I can give a few examples:

1). Engineering teams: When they encounter errors or outages in their programs, Glean is a great debugging tool because the problems they usually face may have already been solved by someone else. Through Glean, they can quickly find relevant content and solutions to help resolve their issues.

2). Support teams: Support teams' daily work is solving people's problems, and Glean fundamentally changes how they work. They no longer need to manually search across different knowledge bases for answers; instead they can directly get answers to questions through Glean. Their main job becomes verifying whether the AI's answer is correct, then sharing it with the customer.

3). Sales teams: They use Glean to prepare for meetings. For example, before a customer meeting, they can quickly ask Glean questions to get complete customer data — what was discussed last time, current opportunities, and so on. This helps them efficiently prepare and handle questions during meetings, boosting confidence. At our company, salespeople don't even need to ask technical support engineers for professional responses, because Glean can already help them answer customer questions based on previous solutions. That's the convenience Glean brings. We find the most expert content on relevant topics based on your question, and this is also a feature we see enterprise users using very heavily.

8. Pat Grady: How do you judge or measure whether these answers are effective? Through very direct user feedback like thumbs-up, or through more indirect user behaviors?

Arvind Jain: We judge through indirect behaviors. For example, when users click the first two or three search results and stay for a significant amount of time, without quickly returning or searching again, we consider that they successfully found their answer. This behavior is our main basis for measuring success.

Part.04

Glean Is Not an AI Company 9. Pat Grady: This isn't your first startup. Rubrik was a huge success, and you played an important role in Google's early growth. How much difference do you think there is between founding a company specifically for AI and just founding a company?

Arvind Jain: I think AI is just a tool. You don't become a completely different company just because you use AI. I believe no new company in the future will entirely avoid AI technology. My view is that entrepreneurs need to find a business problem they want to solve, then use AI technology to make the solution better and more efficient. This approach doesn't change the nature of the company. In fact, we don't even consider ourselves an AI company.

10. Sonya Huang: Do you have plans to train your own models? More broadly, where are Glean's core capability boundaries?

Arvind Jain: We don't have plans to train large-scale models. But at the same time, we do train smaller models. For example, for each customer, we train a dedicated language model. These models deeply analyze the customer's enterprise corpus, understanding terminology, abbreviations, and so on within it.

Model training is a core part of Glean's core technology, but it's not training massive models like GPT-4. We won't enter that domain, and have no plans to. Instead, we'll collaborate with excellent companies building such models, focusing our resources on our core technology and applications.

📮 Further Reading

Linear Bolt Bolt is Linear Capital's dedicated investment program for early-stage, global-market-facing AI applications. It upholds Linear's investment philosophy, focusing on projects where technology-driven transformation creates change, with the goal of helping founders find the shortest path to achieving their objectives. Whether in speed of action or investment approach, Bolt's commitment is lighter, faster, and more flexible. In the first half of 2024, Bolt invested in seven AI application projects including Final Round, Xinguang (心光), Cathoven, Xbuddy, and Midreal.