Guanaco, an open source chatbot created by researchers at the University of Washington, aims to rival ChatGPT while requiring less money and time for training.
Guanaco, an open source chatbot developed by University of Washington researchers, attempts to compete with ChatGPT’s performance while using a great deal less time and resources during training. Guanaco, named after a llama relative from South America, is based on the LLaMA language model and uses a brand-new fine-tuning technique called QLoRA.
Guanaco’s developers assert that it performs on par with ChatGPT while taking just one day to train. This amazing accomplishment is made possible by QLoRA, a method for fine-tuning language models that significantly lowers the amount of GPU RAM required for training. Guanaco’s most basic version only needs 5 GB of GPU RAM, compared to ChatGPT’s staggering 780 GB requirement for a model with 65 billion parameters.
Guanaco and other such open source models are challenging the idea that costly training is required for cutting-edge language models with these spectacular efficiency benefits. The future of expensive models like GPT has been questioned in light of the rise of models like Guanaco, Alpaca, and others that train for a small fraction of the price.
Not everyone shares this upbeat opinion of open source models, though. Alpacas are used as models, but a recent study by the University of California has called into question their abilities and their genuine potential. The researchers first came to the same conclusion as the designers of Guanaco: when properly trained, open source models may compete with GPT in terms of capability. Additional testing uncovered a substantial constraint. These “Dolly” models, as they are commonly referred as, are skilled at modelling fixes for issues they’ve come across throughout training. They fall short of more sophisticated models when it comes to tasks they haven’t been formally exposed to.
This information raises the possibility that the millions spent on training GPT and related models weren’t for nothing. Although Guanaco and its competitors show encouraging results, more complex models continue to perform better in some instances. It is important to note that the University of California’s research casts doubt on the widely held belief that cheap models like GPT may completely be replaced by open-source alternatives.
It will be fascinating to see how Guanaco and other open source models fare against reputable benchmarks like ChatGPT as the area of natural language processing develops. The future of language models will undoubtedly be impacted by the high rate of innovation and ongoing research, which will also define which models are preferred for particular applications.
