Author: Zach Winn | MIT News<\/p>\n
The impact of artificial intelligence will never be equitable if there\u2019s only one company that builds and controls the models (not to mention the data that go into them). Unfortunately, today\u2019s AI models are made up of billions of parameters that must be trained and tuned to maximize performance for each use case, putting the most powerful AI models out of reach for most people and companies.<\/p>\n
MosaicML started with a mission to make those models more accessible. The company, which counts Jonathan Frankle PhD \u201923 and MIT Associate Professor Michael Carbin as co-founders, developed a platform that let users train, improve, and monitor open-source models using their own data. The company also built its own open-source models using graphical processing units (GPUs) from Nvidia.<\/p>\n
The approach made deep learning, a nascent field when MosaicML first began, accessible to far more organizations as excitement around generative AI and large language models (LLMs) exploded following the release of Chat GPT-3.5. It also made MosaicML a powerful complementary tool for data management companies that were also committed to helping organizations make use of their data without giving it to AI companies.<\/p>\n
Last year, that reasoning led to the acquisition of MosaicML by Databricks, a global data storage, analytics, and AI company that works with some of the largest organizations in the world. Since the acquisition, the combined companies have released one of the highest performing open-source, general-purpose LLMs yet built. Known as DBRX, this model has set new benchmarks in tasks like reading comprehension, general knowledge questions, and logic puzzles.<\/p>\n
Since then, DBRX has gained a reputation for being one of the fastest open-source LLMs available and has proven especially useful at large enterprises.<\/p>\n
More than the model, though, Frankle says DBRX is significant because it was built using Databricks tools, meaning any of the company\u2019s customers can achieve similar performance with their own models, which will accelerate the impact of generative AI.<\/p>\n
\u201cHonestly, it\u2019s just exciting to see the community doing cool things with it,\u201d Frankle says. \u201cFor me as a scientist, that\u2019s the best part. It\u2019s not the model, it\u2019s all the amazing stuff the community is doing on top of it. That’s where the magic happens.\u201d<\/p>\n
Making algorithms efficient<\/strong><\/p>\n Frankle earned bachelor\u2019s and master\u2019s degrees in computer science at Princeton University before coming to MIT to pursue his PhD in 2016. Early on at MIT, he wasn’t sure what area of computing he wanted to study. His eventual choice would change the course of his life.<\/p>\n Frankle ultimately decided to focus on a form of artificial intelligence known as deep learning. At the time, deep learning and artificial intelligence did not inspire the same broad excitement as they do today. Deep learning was a decades-old area of study that had yet to bear much fruit.<\/p>\n \u201cI don\u2019t think anyone at the time anticipated deep learning was going to blow up in the way that it did,\u201d Frankle says. \u201cPeople in the know thought it was a really neat area and there were a lot of unsolved problems, but phrases like large language model (LLM) and generative AI weren\u2019t really used at that time. It was early days.\u201d<\/p>\n Things began to get interesting with the 2017 release of a now-infamous paper<\/a> by Google researchers, in which they showed a new deep-learning architecture known as the transformer was surprisingly effective as language translation and held promise across a number of other applications, including content generation.<\/p>\n In 2020, eventual Mosaic co-founder and tech executive Naveen Rao emailed Frankle and Carbin out of the blue. Rao had read a paper the two had co-authored, in which the researchers showed a way to shrink deep-learning models without sacrificing performance. Rao pitched the pair on starting a company. They were joined by Hanlin Tang, who had worked with Rao on a previous AI startup that had been acquired by Intel.<\/p>\n The founders started by reading up on different techniques used to speed up the training of AI models, eventually combining several of them to show they could train a model to perform image classification four times faster than what had been achieved before.<\/p>\n \u201cThe trick was that there was no trick,\u201d Frankle says. \u201cI think we had to make 17 different changes to how we trained the model in order to figure that out. It was just a little bit here and a little bit there, but it turns out that was enough to get incredible speed-ups. That\u2019s really been the story of Mosaic.\u201d<\/p>\n The team showed their techniques could make models more efficient, and they released an open-source large language model in 2023 along with an open-source library of their methods. They also developed visualization tools to let developers map out different experimental options for training and running models.<\/p>\n MIT\u2019s E14 Fund invested in Mosaic\u2019s Series A funding round, and Frankle says E14\u2019s team offered helpful guidance early on. Mosaic\u2019s progress enabled a new class of companies to train their own generative AI models.<\/p>\n \u201cThere was a democratization and an open-source angle to Mosaic\u2019s mission,\u201d Frankle says. \u201cThat\u2019s something that has always been very close to my heart. Ever since I was a PhD student and had no GPUs because I wasn\u2019t in a machine learning lab and all my friends had GPUs. I still feel that way. Why can\u2019t we all participate? Why can\u2019t we all get to do this stuff and get to do science?\u201d<\/p>\n Open sourcing innovation<\/strong><\/p>\n Databricks had also been working to give its customers access to AI models. The company finalized its acquisition of MosaicML in 2023 for a reported $1.3 billion.<\/p>\n \u201cAt Databricks, we saw a founding team of academics just like us,\u201d Frankle says. \u201cWe also saw a team of scientists who understand technology. Databricks has the data, we have the machine learning. You can’t do one without the other, and vice versa. It just ended up being a really good match.\u201d<\/p>\n In March, Databricks released DBRX, which gave the open-source community and enterprises building their own LLMs capabilities that were previously limited to closed models.<\/p>\n \u201cThe thing that DBRX showed is you can build the best open-source LLM in the world with Databricks,\u201d Frankle says. \u201cIf you\u2019re an enterprise, the sky\u2019s the limit today.\u201d<\/p>\n Frankle says Databricks\u2019 team has been encouraged by using DBRX internally across a wide variety of tasks.<\/p>\n \u201cIt\u2019s already great, and with a little fine-tuning it\u2019s better than the closed models,\u201d he says. \u201cYou\u2019re not going be better than GPT for everything. That\u2019s not how this works. But nobody wants to solve every problem. Everybody wants to solve one problem. And we can customize this model to make it really great for specific scenarios.\u201d<\/p>\n As Databricks continues pushing the frontiers of AI, and as competitors continue to invest huge sums into AI more broadly, Frankle hopes the industry comes to see open source as the best path forward.<\/p>\n \u201cI\u2019m a believer in science and I\u2019m a believer in progress and I\u2019m excited that we\u2019re doing such exciting science as a field right now,\u201d Frankle says. \u201cI\u2019m also a believer in openness, and I hope that everybody else embraces openness the way we have. That’s how we got here, through good science and good sharing.\u201d<\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"