Blog
Company Updates & Technology Articles
March 7, 2024
Introducing WMDP: Measuring and Mitigating Catastrophic Risk Potential from LLMs
In partnership with the Center for AI Safety, Scale is proud to publish a novel safety evaluation benchmark for large language models: the Weapons of Mass Destruction Proxy (WMDP).
Read more
February 20, 2024
Scale AI Partners with DoD’s Chief Digital and Artificial Intelligence Office (CDAO) to Test and Evaluate LLMs
Scale AI, the leading test and evaluation (T&E) partner for frontier artificial intelligence companies, is proud to share that we are partnering with the U.S. Department of Defense’s (DoD) Chief Digital and Artificial Intelligence Office (CDAO) to create a comprehensive T&E framework for the resp...
Read more
February 13, 2024
Accelerate Generative AI Across Your Enterprise with Scale GenAI Platform
2023 ushered in a wave of excitement about Large Language Models (LLMs) and became the year of the Generative AI proof-of-concept. Enterprises experimented with Generative AI and explored how it may impact their business. According to BCG,
Read more
February 8, 2024
Scale AI Joins U.S. Artificial Intelligence Safety Institute Consortium
Scale AI is proud to announce a new collaboration with the National Institute of Standards and Technology (NIST) in the Artificial Intelligence Safety Institute Consortium (AISIC) to develop science-based and empirically backed guidelines and standards for AI....
Read more
January 30, 2024
Unraveling the Mysteries of Inter-Rater Reliability
Imagine you have submitted a research paper to a leading conference in the field of AI. Several reviewers will assess your work, each providing a rating from a set of four categories: accept, weak accept, weak reject, and reject.
Read more
January 24, 2024
2024: The Year of AI Implementation and Legislation
As we step into 2024, it is clear that artificial intelligence (AI) will continue to dominate global government discussions. Last year saw over 100 new requirements for the federal governments from the Executive Order, OMB implementation memo, and NDAA....
Read more
December 21, 2023
Scale AI and Austin Community College Host First Public Sector Generative AI Hackathon
Scale AI and Austin Community College District (ACC) recently teamed up to host a hackathon that enabled participants to craft prototypes with practical applications using Donovan, Scale’s AI-powered digital staff assistant. The hackathon, held on December 12 at the ACC Rio Grande Campus ACCelerator
Read more
December 12, 2023
Efficient and Effective Fine-Tuning Using Mixture-of-Experts PEFT
At Scale, we have always believed that building custom LLMs through fine-tuning is key to unlocking greater performance for any given organization’s specific use case. We work with enterprise customers to implement cutting-edge enterprise Generative AI solutions, combining the best large language...
Read more
December 6, 2023
We Fine-Tuned GPT-4 to Beat the Industry Standard for Text2SQL
Our machine learning team at Scale has recently fine-tuned GPT-4 to achieve state-of-the-art performance (84% accuracy) for generalized text-to-SQL translation on one of the most popular benchmark datasets, the SpiderDev Set. In this blog post, we will discuss why text2sql is an important use case..
Read more
December 5, 2023
Introducing Scale’s Automotive Foundation Model
Autonomous vehicle development requires iterative improvements in perception models through a data engine. These data engines currently rely on a set of task-specific models based around a fixed taxonomy of objects and scenarios to identify. However, there are two critical limitations to existing...
Read more