The World's Fastest AI Inference Solution: Leaving NVIDIA in the Dust by 20x

🚀 World's Fastest AI Inference Solution

🌏 Anthropic Opens Artifacts AI Feature to All Users

👥 NVIDIA Launches NIM Agent Blueprints Service

🌐 Zhipu AI Opens GLM-4-Flash Large Language Model to Public

🛡️ New Facial Privacy Protection Solution: FaceObfuscator

💻 Quark Releases All-New PC Client

🤝 Japanese Financial Giant SBI Forms Alliance with Chip Startup PFN

📈 Enflame Valuation Exceeds 16 Billion RMB

🦟 Bill Gates Leverages AI to Combat Malaria

World's Fastest AI Inference Solution

Cerebras Systems has announced the launch of Cerebras Inference, billed as the world's fastest AI inference solution. Cerebras Inference delivers 1,800 tokens per second for the Llama 3.1 8B model and 450 tokens per second for the Llama 3.1 70B model — 20 times faster than NVIDIA GPU-based solutions.

Cerebras Inference is priced far below popular GPU cloud services, starting at 10 cents per million tokens and offering 100x better price-performance. The solution uses Cerebras CS-3 systems and the Wafer Scale Engine 3 (WSE-3) AI processor; the CS-3's memory bandwidth is 7,000 times that of the NVIDIA H100, effectively solving the memory bandwidth bottleneck that has constrained generative AI. Cerebras Inference sets a new standard for open large language model (LLM) development and deployment, and is the only solution capable of delivering both high-speed training and inference.

Anthropic Opens Artifacts AI Feature to All Users

AI company Anthropic announced that its Artifacts feature is now available to all Claude users, who can create and browse Artifacts in the Claude iOS and Android apps. Artifacts functions as a dynamic workspace, allowing users to generate documents, code, vector graphics, and more during conversations with Claude, with the ability to view and iterate on their work in real time.

Artifacts launched in preview in June 2024; to date, users have created tens of millions of Artifacts. With a single tap, users can open Artifacts in a dedicated window alongside their conversation, streamlining workflows and improving communication efficiency among team members.

NVIDIA Launches NIM Agent Blueprints Service

NVIDIA announced the launch of its new NIM Agent Blueprints service in partnership with global collaborators, helping enterprises build their own AI experiences. NIM Agent Blueprints is a catalog of pre-trained, customizable AI workflows — a complete software suite that enables enterprise developers to build and deploy generative AI applications. Through NIM Agent Blueprints, enterprises can create a data-driven AI flywheel for continuous optimization and performance improvement.

The NIM Agent Blueprints service covers a range of typical use cases, including customer service avatars, retrieval-augmented generation, and virtual screening for drug discovery. Developers can use one or more AI agents to create applications, with sample apps built on NVIDIA NeMo, NVIDIA NIM, and partner microservices. Tools such as Helm charts make it easy for enterprises to deploy generative AI applications in accelerated data centers and clouds. Enterprises can customize NIM Agent Blueprints using their own business data and refine AI applications based on user feedback.

Zhipu AI Opens GLM-4-Flash Large Language Model to Public

Zhipu AI announced that its GLM-4-Flash large language model[1] is now freely available to the public and accessible through the Zhipu AI open platform. GLM-4-Flash is designed for straightforward vertical tasks, low-cost requirements, and rapid response, with a generation speed of 72.14 tokens per second — approximately 115 characters per second.

The GLM-4-Flash model supports multi-turn conversations, web browsing, function calling, and long-text reasoning (with a maximum context length of 128K), and covers 26 languages including Chinese, English, Japanese, Korean, and German. It employs adaptive weight quantization, multiple parallelization methods, batching strategies, and speculative sampling to reduce latency and boost inference speed, achieving greater concurrency and throughput, improving efficiency, and lowering costs.

New Facial Privacy Protection Solution: FaceObfuscator

To address the threat of facial feature reconstruction attacks in facial recognition systems, Zhejiang University and Alibaba's security team jointly developed a new facial privacy protection solution — FaceObfuscator[2]. FaceObfuscator filters frequency-domain channels to remove redundant visual information from facial images and introduces randomized interference to effectively defend against reconstruction attacks, while the server removes the randomness through inverse transformation to maintain facial recognition accuracy.

FaceObfuscator was tested on six public facial datasets. Experimental results show that it cannot be reconstructed into facial images, effectively protecting facial privacy. It performs excellently on COS (cosine similarity) and SRRA (replay attack success rate) metrics, significantly reducing the success rate of reconstruction attacks. FaceObfuscator can be widely applied in scenarios such as surveillance identification, face-scan payments, and access control attendance, serving critical industries including security, finance, and education, and helping to solve facial privacy security challenges.

Quark Releases All-New PC Client

Quark, under Alibaba's intelligent information business group, has released a new PC client that transforms ordinary computers into AI PCs through system-level, full-scenario AI capabilities, providing one-stop information retrieval, creation, and summarization services.

AI search, AI writing, AI PPT, and AI document summarization have all been comprehensively upgraded, significantly improving information processing efficiency. Quark's AI answers lead the industry in both speed and accuracy. Its three-column interface design presents information clearly, and AI functions can be called up anytime via keyboard shortcuts, simplifying search, interpretation, and translation operations. Right-clicking on a document instantly summarizes key information and converts document formats.

Japanese Financial Giant SBI Forms Alliance with Chip Startup PFN

Preferred Networks (PFN), a "unicorn" in Japan's AI chip sector, and financial giant SBI Holdings announced a capital and business alliance for the development and commercialization of PFN's next-generation AI semiconductors. The two parties plan to jointly develop PFN's next-generation AI chips and collaborate on bringing them to market, with cooperation extending to chip packaging and testing processes.

The SBI Group plans to invest in PFN through SBI Holdings, with an investment of up to 10 billion yen (approximately 494 million RMB) to be completed by the end of September 2024. PFN's next-generation AI chip is expected to use Samsung Electronics' 2nm process and I-Cube S advanced packaging technology to produce AI accelerators.

Enflame Valuation Exceeds 16 Billion RMB

Enflame, a leading company in China's AI chip industry, has officially initiated IPO listing counseling and is poised to become the second AI computing design company on the STAR Market. Tencent is Enflame's single largest shareholder with a 20.4944% stake, while Lidong Zhao and Yalin Zhang collectively control 32.5087% of the company's voting rights through direct and indirect shareholdings.

Founded in March 2018, Enflame focuses on AI computing products. It has released two generations of training and inference products spanning chips, accelerator cards, intelligent computing all-in-one machines, liquid-cooled computing clusters, and supporting software systems. With comprehensive original innovation and R&D capabilities, it ranks among the top domestic players in the AI computing field. Enflame serves more than 20 internet customers and over 50 enterprise customers, with more than 500 deployed scenarios across finance, internet, AI large models, and other domains.

Bill Gates Leverages AI to Combat Malaria

Microsoft founder Bill Gates announced significant progress in using computer vision technology to fight malaria. In a blog post, Gates introduced VectorCam, an app that can rapidly identify mosquito species, developed by Dr. Sumiya Achariya and her team at Johns Hopkins University. Gates also mentioned HumBug, a technology that identifies mosquito species based on the sound of their wingbeats, which, though still in early development, holds promise for automated continuous monitoring.

Supported by the Gates Foundation and the Uganda Malaria Control Program, VectorCam can distinguish mosquito species, sex, and whether they have fed blood or laid eggs within seconds using just a smartphone and an inexpensive lens. VectorCam streamlines the process of mosquito data collection, identification, and reporting, enabling local health workers to participate more efficiently in surveillance and allowing disease prevention personnel to focus on broader strategic tasks.

Gates believes that while identifying mosquito species is crucial, better tools must be developed to eradicate malaria. He remains optimistic about these innovative technologies and believes they will help bring the world closer to the goal of malaria elimination.

We come into this world to search for answers. Not that you'll necessarily find them — but you have to try.

— What to Do When I'm Gone

Editorial Team Editor: Yuki

Design: Ivan

For business partnerships, add on WeChat: Rwkfbcianvd

References [1] GLM-4-Flash large language model: https://bigmodel.cn/console/trialcenter?modelCode=glm-4-flash

[2] FaceObfuscator: https://www.usenix.org/conference/usenixsecurity24/presentation/jin-shuaifan