Palantir: How the Secretive Data Analytics Giant Won in the GenAI Era

十字路口Crossing十字路口Crossing··1·0

The smartest people, the most cutting-edge technology, the hardest problems.

🟢 Author: "Crossing" AI Research Fellow @Jing

📌 Preface: Palantir is a publicly traded company specializing in big data analytics and decision intelligence, with close business ties to the United States government. It is rumored to have played a significant role in famous American military operations such as the killing of Osama bin Laden. Secret deals with the U.S. military, the political leanings of its founding team, and data privacy and security concerns have drawn considerable controversy to Palantir. After the April 2023 launch of AIP, a new product built on GenAI, Palantir's soaring stock price has attracted even more attention. This article briefly reviews Palantir's entrepreneurial journey, focuses on the product advantages and commercialization approach of AIP, and concludes by introducing the unique management team behind this secretive giant.

Origin Story: The Palantír, Impossible to Value

The Smartest People, the Best Technology, the Hardest Problems

Palantir[1] was founded in 2003. Four of its five co-founders came from the "PayPal Mafia," including godfather figure Peter Thiel. (Note: "PayPal Mafia" commonly refers to a group of early key figures at PayPal who went on to found multiple successful tech companies after leaving PayPal and profoundly influenced Silicon Valley's venture capital culture. They include Peter Thiel, Elon Musk, and J.D. Vance, who was recently nominated as the Republican vice-presidential candidate.) Palantir's name derives from a mysterious and powerful seeing-stone in The Lord of the Rings, reflecting the founding team's hope that Palantir would be able to perceive the past and foresee the future.

The villainous wizard Saruman using the "Palantír" to spy on his enemies in The Lord of the RingsIn the anxious atmosphere of the post-9/11 era, Peter led the team in transferring PayPal's accumulated anti-fraud expertise to the counterterrorism domain, dedicated to assembling the world's smartest people, leveraging the most advanced data analytics and AI technology, and solving the world's most difficult problems. In 2004, after self-funding the development of its first-generation product prototype, Palantir quickly secured seed funding from the CIA's venture capital arm[2] and began piloting its product at the agency's core institutions. From 2005 to 2008, the CIA was Palantir's sole sponsor and only customer[3], deeply refining Palantir's product and laying an important foundation for Palantir's later success in the U.S. government market.

With its government business taking initial shape, Palantir brought its technical expertise to Wall Street[4]. Financial trading is fundamentally about pricing risk, and effective pricing depends on precise, efficient integration and analysis of multi-source heterogeneous data. Risk prevention and control during the trading process similarly requires real-time computation on massive datasets with visualized monitoring — precisely Palantir's strengths. In 2010, introduced by its New York Police Department client, Palantir partnered with JPMorgan and officially entered the financial markets, becoming Palantir's first commercial client. According to a former JPMorgan employee[5], "These tools have saved the firm hundreds of millions of dollars, addressing problems ranging from cyber fraud to bad mortgages. A JPMorgan user of Palantir's software can, within seconds, see connections between a Nigerian IP address, a proxy server somewhere in America, and a payment flowing out of a hijacked home-equity line of credit — just as military clients piece together fingerprints on bomb fragments, location data, anonymous tips, and social media messages to track Afghan bomb makers."

From Big Data Counterterrorism to GenAI Commercial Applications

At its 2020 IPO, Palantir had established two mature product lines[6]: Gotham, launched in 2008, and Foundry, launched in 2016.

Gotham was Palantir's early core business, primarily serving U.S. government and military institutions including the National Security Agency (NSA), Federal Bureau of Investigation (FBI), Central Intelligence Agency (CIA), and the U.S. Army, Navy, Air Force, and Marines. This is the main reason Palantir appears so mysterious. The name "Gotham" is itself interesting: in DC Comics, "Gotham City" is a city shrouded in darkness, crime, and corruption, the primary backdrop for Batman's vigilante justice. Amid the treacherous landscape of terrorism and war, Palantir hoped that Gotham would provide "justice-wielding" government agencies with more efficient tools for intelligence mining, situational analysis, risk monitoring, and decision support.

Gotham City in Batman comicsFoundry is Palantir's platform product for the commercial sector, designed to provide large institutions with unified data management systems for enterprise-level, full-lifecycle data management, operations, and decision support. To balance the risk of over-reliance on government business, Palantir used Foundry to continuously develop commercial clients, successfully expanding from financial institutions into energy, transportation, biopharmaceuticals, consumer goods, and other industries.

By 2019, driven by both Gotham and Foundry, Palantir had generated $740 million in revenue, with commercial revenue surpassing government revenue at 53% of the total.

Palantir Revenue Growth Trend, 2008–2019Palantir subsequently launched Apollo, a CI/CD platform that enables Palantir's product technology to achieve rapid deployment and convenient operations and maintenance across the complex environments of different clients.

On April 7, 2023, several months after ChatGPT's debut, Palantir officially released its fourth platform product — AIP (Artificial Intelligence Platform). This was Palantir's first product to deeply apply GenAI technology, designed to provide large institutions with a platform for rapidly building and applying AI products based on LLMs and converting them into operational outcomes.

AIP's launch kicked off Palantir's sustained growth in performance and stock price over the past year-plus. In 2023, Palantir recorded $2.23 billion in revenue, up 17% year-over-year (with commercial revenue up 20%), and achieved full-year net profit of $210 million — its first profitable year since going public[7]. By July 2024, Palantir's market capitalization had exceeded $63 billion, with a total return of over 240% since AIP's launch, far outpacing other software companies of comparable scale. Palantir's current P/E ratio exceeds 80x, which many analysts consider "beyond reasonable range."

Palantir and Comparable Companies' Price Return (April 7, 2024–July 19, 2024)

AIP's Product Advantages and Commercialization Approach

How AIP Leverages Palantir's Traditional Strengths

"Garbage in, garbage out." This is a common saying among AI practitioners. Whether from the perspective of model training or big data analytics applications, the quality of input data often has a decisive impact on model performance and decision quality — and the same holds true for AI applications in the large model era. Take RAG, for example: we know the core of RAG is the "R" (retrieval), and the key to R is whether the data in the knowledge base has been built and enhanced with high-quality standardization oriented toward the business domain.

Palantir has accumulated 20 years of experience in the military domain, where data complexity is extraordinarily high, and possesses a deep understanding of how data processing pipelines, domain modeling, and human-computer interaction should connect from raw data to decision support through product design. This has endowed its AIP product with inherent advantages at both the data layer and application layer.

We can get a sense of how AIP operates[8] through an official military-domain example published by Palantir:

The example begins from the perspective of an officer responsible for monitoring military activity in Eastern Europe. He has just received an alert showing military equipment massing at a location 30 kilometers from friendly forces. The officer quickly poses a question to AIP Assistant (note AIP's response in the screenshot):

  • Show me more details.

  • What military unit is in the region?

  • Show options for tasking imagery at this location at a resolution of 1 meter or higher. (Help me find options to capture clearer imagery of this location)

  • For the third query, the AIP Assistant provided an option to deploy a drone. The officer entered the command: "Task the MQ-9 to capture video of this location." After the command was issued, the live video feed from the drone confirmed the presence of an enemy main battle tank in the target area.

  • The officer then entered: "Generate 3 courses of action to target this enemy equipment." — AIP quickly responded with tactical recommendations, listing the resources and time required for each. The officer followed up with: "Send these three options to my commander for review." — AIP promptly forwarded the recommendations to the commander for approval.

  • The video revealed the backend logic behind this seamless workflow: as the officer asked questions, AIP traversed and analyzed real-time data collected from multiple sources — data that was cleaned, tagged, mapped to analytical objects, and subject to strict access controls. This data then fed into traditional models and analytical systems, with AIP invoking the appropriate system tools (tool calls) based on the user's requests and synthesizing the results. For every analytical conclusion, AIP maintained full traceability to its underlying data and system tools.

After demonstrating the junior officer's workflow, the example switched to the commander's perspective — upon receiving the three courses of action for review, the commander ultimately selected Option Three. In subsequent interactions, AIP assisted the commander in developing a detailed operational plan, including route planning using geospatial intelligence and military unit disposition, ammunition supply preparation, and an enemy communications jamming plan. The interface interactions were not shown in full; interested viewers are strongly encouraged to watch the original video.

  • Notably, in the backend logic shown in this section, we can clearly see that such a complex systems engineering feat is far from being accomplished by a large language model alone. It relies on Palantir's long-accumulated expertise in complex data processing, including mature data pipelines and domain-specific small models trained with traditional algorithms (see the Model Library in the diagram). The role of the LLM in AIP is more that of an intelligent hub that strings together Palantir's deep historical capabilities, leveraging techniques such as RAG, tool calls, and prompt engineering (see the GPT-NeoX model management menu in the AIP Control Panel) to deliver impressive performance in Palantir's traditional areas of strength.

AIP and Ontology

Palantir breaks down enterprise decision-making into three core elements: Data, Logic, and Action. Together these constitute Palantir's enterprise decision model — Ontology — which is also the foundation of the AIP product. Relative to the better-known RAG concept, Palantir creatively proposes OAG (Ontology Augmented Generation), intended to convey that AIP's advantage lies not merely in knowledge and data retrieval, but in simultaneously enhancing large model capabilities with enterprise private data (Data), internal system tools (Logic), and operational feedback from business personnel (Action) to achieve business optimization. Specifically:

  • At the Data layer, Palantir's strength lies in deep business understanding, enabling effective abstraction and refinement of the "nouns" of the enterprise, and based on this, normalizing multi-source heterogeneous data into unified semantic representations (human interfaces) and service interfaces (AI interfaces).

  • At the Logic layer, this manifests primarily as the combination of large models with traditional small models, integration with business systems and logic (through tool calls), and low-code/no-code workflow orchestration tools that effectively tie these together. This is likely where AIP most resembles common agent platforms.

  • At the Action layer, Palantir is explicit that LLMs supplement rather than replace human action. Therefore, AIP's Workflow Services are designed with complete controls, reviews, and simulation procedures, including isolated sandbox environments and human-in-the-loop feedback, to ensure AI-generated action recommendations can be executed safely, with human feedback continuously iterating and updating the models and processes at the Logic layer.

In the military scenario example described earlier, Palantir's capabilities at the Data layer were demonstrated to full effect — whether in raw data acquisition and processing or in presenting results, with support for multimodal and real-time functionality. At the Logic layer, AIP's strong performance relied on Palantir's long-standing accumulation in the defense sector, enabling rapid invocation of traditional models and analytical tools, including core capabilities from Palantir's legacy Gotham product. At the Action layer, we see real-time interaction between users and AIP, and between AIP and system operations, but more control mechanisms operate behind the scenes, including access controls, content review, and data provenance.

Built on Ontology and Palantir's historical foundation, the AIP platform offers customers two choices: using out-of-the-box AI applications (AIP Now), or building custom AI applications (Build with AIP).

On Palantir's AIP Now product page, you can find hundreds of application examples, categorized by industry and function — this is practically a product-market fit map for large language model applications, where virtually any scenario might correspond to a startup. This is also a testament to Palantir's years of accumulated business know-how across government and commercial sectors.

Here's another example from finance: in primary equity investment, company screening and due diligence has always been a time-consuming, labor-intensive process. The video mentions one VC firm that reviews over 7,000 pitch decks per year, of which 1,000 advance to due diligence, 250 to deep investigation and engagement, and only 20 receive investment. Throughout this drawn-out process with a conversion rate below 0.3%, AIP can assist investment teams across the full workflow — document review and screening, due diligence questionnaire generation and response analysis, investment committee report preparation — while enabling progress visualization and data traceability. This significantly boosts operational efficiency, allowing investment teams to focus on truly high-value companies while "finding a needle in a haystack." In this example, we again see Palantir's data processing capabilities, its ability to orchestrate business workflows and integrate tools, and its product implementation of human-machine interaction.

Bootcamps: Cultivating a New Customer in Five Days

For any company commercializing large language model products for enterprise customers, a common challenge arises — how do you get clients to recognize and value the foundational infrastructure potential of large models, rather than evaluating LLM projects solely through isolated, fragmented use cases measured by ROI?

Palantir chose to lead by example, completing customer education and sales through ultra-short-term POC-style bootcamps.

These 1- to 5-day AIP bootcamps have generated substantial business opportunities for Palantir: as of February 2024, Palantir had completed 560 bootcamps for 465 customers, with another 450 customers added between February and July. Palantir management stated that AIP bootcamps are "rapidly converting to paid customers" — a leading utility company signed a seven-figure contract just five days after completing a bootcamp, while another customer signed a paid agreement one day after a multi-day bootcamp and converted to a seven-figure deal three weeks later. In Q1 2024, Palantir's U.S. commercial bookings grew 94% year-over-year to 136 deals, with total contract value (TCV) up 131% year-over-year to $286 million[9].

We are good at educating customers on what is the art of the possible, and then some portion of those customers buy it. — Alex Karp, CEO of Palantir

The "Artist Colony" Behind the Success

To close, I'd like to spend some time on Palantir's distinctive management team.

CEO Alex Karp: An "Iconoclastic" Philosopher

I view building a business as an art form. I believe that if you change 'artists' into a 'I-want-to-do-it-in-a-creative-way' endeavor, it means I'm going to build something not in the way I was taught in school. However, it addresses the underlying issues of your art. — Alex Karp, CEO of Palantir (source)[10]

Like Peter Thiel, CEO Alex Karp earned his JD from Stanford University, but unlike Peter, he pursued deeper training in the humanities — after leaving Stanford, Alex didn't practice law, but instead went to Goethe University Frankfurt to continue his doctoral studies in Neoclassical Social Theory[11]. Alex became absorbed in researching the philosophical foundations of Western social construction, but gradually discovered how limited academic research could be in the real world, and began pivoting toward business. He eventually reconnected with his old Stanford acquaintance Peter and joined Palantir's founding team as CEO[12]. Peter's willingness to invite Alex, who had no technical background whatsoever, to become a co-founder followed his consistent "meritocracy regardless of background" philosophy from his PayPal days.

Alex has been described by media as an "iconoclastic" philosopher. He has cited Freud to argue that social violence doesn't stem entirely from economic factors, but from destructive impulses deeply embedded in human nature — and therefore such risks cannot be captured through structured economic analysis alone, but also require "finding traces of unconscious aggression through parsing language, revealing hidden identities and kinship relations, and the violent motives concealed within them"[13]. When articulating Palantir's mission, Alex frequently references Hegelian dialectics; Palantir has consistently sought to aggregate opposing data and viewpoints, uncovering deeper connections to approach "higher truth." And whether in big data or AI applications, Palantir firmly believes that rational decision-making requires the combination of human thinking and machine intelligence — machine intelligence can only discover patterns in formal logic, which is insufficient to address "adaptive" human opponents capable of transcending formal logic. This also aligns with Hegel's emphasis on "practical reason."

Stemming from his own deep connection to art, Alex views business itself as an art form. He opposes narrowly defining art as painting, writing, music, and similar activities; in a broader sense, art is a habit of thinking that creatively deconstructs and solves complex problems from their essence[14]. Perhaps it is precisely this habit of thinking that has enabled a philosopher to lead Palantir's world-class engineering team to sustained success in the complex business world.

CTO Shyam Sankar: Technology Is a One-Way Ticket Out of the Third World

Instead, you want to be an artist colony. What does managing and leading 10 of the world's greatest artists look like? It isn't command and control. It doesn't hew to any structure or strictures. Instead, the leader must maximize the unique strengths of each individual in the colony — with the right person in the right role in the right time — to produce a great work of art. — Shyam Sankar, CTO of Palantir (source)[15]

Shyam Sankar was the company's 13th employee and currently serves as CTO. Born in Mumbai, India, Shyam spent a turbulent early childhood in Nigeria — at age two, armed intruders broke into their home and injured his father. After the family relocated to Orlando, Florida, his parents' entrepreneurial journey remained filled with hardships. Shyam first encountered technology in high school and developed an intense passion for software. He later attended Cornell University and Stanford University, officially joining Palantir in 2006.

In April 2024, Shyam published a long essay in Pirate Wires[16] sharing his understanding of high-performance teams. He emphasized internalizing the pursuit of "winning" into organizational culture, creating an "artist colony" that fully unleashes each individual's unique talents, embraces chaos, and drives self-motivated growth — rather than a mercenary factory held together by management systems. To this end, Palantir pays below-market salaries but offers generous equity. Shyam noted in his article, "We will be a cult. Joining will be painful, so applicants must really want to be here."

Shyam pinned this article to his Twitter profile, reiterating in the accompanying post another point from his essay:

Engineers get caught up in how they want the world to work, how they've been taught the world should work, but they rarely understand how the world actually works. And you can't win if you aren't living in reality. It's a natural human tendency to define the problem based on what you can win at, but that is the ultimate corruption. You have to take the red pill and see how far down the rabbit hole goes (18 years in, and I haven't hit bottom yet).

工程师们往往沉迷于他们希望世界如何运作，以及他们被教导的世界应该如何运作，但他们很少理解世界实际上是如何运作的。如果你不活在现实中，你是无法获胜的。人类有一个自然的倾向，就是根据自己擅长的领域来定义问题，但那是最终的腐败。你必须服下红色药丸，看看兔子洞究竟有多深（已经18年了，我还没到达底部）。

This is almost a perfect articulation of the cognitive bias that tech idealists hold about the real world. To prevent the company's technical elite from getting lost in the self-indulgent fantasy of "saving the world," Shyam proposed the "Forward Deployed Engineering" model. He required engineers to get their hands dirty in frontline project deployment and delivery — which could mean catching the last military flight of the day to Afghanistan to deliver computer equipment.

Palantir quality engineer Mark Scianna en route to Afghanistan, 2010

After reading Shyam's essay, I was reminded of that line from Game of Thrones: "Chaos is the ladder." With an almost religious force, Palantir has assembled a group of extraordinarily capable, complex, and perhaps deeply flawed individuals in the absence of organizational norms — and has continuously built the formidable commercial position it holds today.

Closing Thoughts

Wittgenstein said that the limits of our language shape the limits of our world. The emergence of large language models has brought the language of the silicon world infinitely closer to humanity's symbolic system. We no longer need to understand machine language to interact with it directly. But what comes after that?

If data can lead directly to truth, is there still a need for translation through human symbols? As in the Ontology that Palantir has built — the "logic" bridge between Data and Action — how much room will there be for human participation in the future?

Hannah Arendt divided human activity into "labor," "work," and "action." My vision for a better world is one where AI replaces human labor and work, while humans need to create new spaces to carry our active "action."

This article comes from Crossing AI Research Fellow @Jing.

About the Author: Jing

Works on financial industry solutions at a large model company

Xiaohongshu: Zhe Zhi's Study and the Sea

WeChat: BC369_ZZ

Crossing AI Research Fellow

In early July, Crossing published a call for "AI Research Fellows". We received nearly 50 applications in total. After carefully reading each candidate's email and materials and chatting with them online, we selected six Fellows for Season 1 (S24). They will conduct research and share their work on AI together with Crossing.

References

[1] Palantir: https://www.palantir.com/

[2] Palantir迅速得到了来自美国CIA风投部门的种子轮融资: https://www.quora.com/How-did-Palantir-gain-its-initial-traction

[3] CIA一直是Palantir的金主和唯一的客户: https://www.forbes.com/sites/andygreenberg/2013/08/14/agent-of-intelligence-how-a-deviant-philosopher-built-palantir-a-cia-funded-data-mining-juggernaut/

[4] Palantir带着技术专长进军华尔街: https://www.quora.com/How-did-Palantir-gain-its-initial-traction

[5] 据JPMorgan的一位前员工称: https://www.forbes.com/sites/andygreenberg/2013/08/14/agent-of-intelligence-how-a-deviant-philosopher-built-palantir-a-cia-funded-data-mining-juggernaut/

[6] Palantir已建立起两条成熟的产品线: https://www.sec.gov/Archives/edgar/data/1321655/000119312520230013/d904406ds1.htm#toc

[7] 全年净利润2.1亿美金，为上市以来首次实现净利润为正: https://investors.palantir.com/news-details/2024/Palantir-Reports-Its-Fifth-Consecutive-Quarter-of-GAAP-Profitability-Fourth-Quarter-GAAP-EPS-of-0.04/

[8] AIP的运作方式: https://www.youtube.com/watch?v=XEM5qz__HOU

[9] 合同总价值（TCV）同比增长131%至2.86亿美元: https://www.forbes.com/sites/bethkindig/2024/07/18/palantirs-stock-is-priced-for-perfection/

[10] (source): https://www.youtube.com/watch?v=3UjvRi6k5Gk&t=326s

[11] Neoclassicial Social Theory: https://outsidertheory.com/the-intellectual-origins-of-surveillance-tech/

[12] 以CEO的角色加入了Palantir的创始团队: https://www.youtube.com/watch?v=-xtuKaM3fEo&t=279s

[13] 「通过解析语言来找到无意识攻击的痕迹，揭示隐藏的身份和亲缘关系，以及潜藏在其中的暴力动机」: https://outsidertheory.com/the-intellectual-origins-of-surveillance-tech/

[14] 艺术是一种用有创意的方式、从本质上解构并解决复杂问题的思考习惯: https://www.youtube.com/watch?v=3UjvRi6k5Gk&t=326s

[15] (source): https://www.piratewires.com/p/primacy-of-winning-shyam-sankar-palantir

[16] Shyam published a long essay in Pirate Wires: https://www.piratewires.com/p/primacy-of-winning-shyam-sankar-palantir