Stability AI Debuts Revolutionary Open-Source Language Model Suite: StableLM

Stability AI has unveiled the first of its cutting-edge open-source language models, StableLM, marking a significant step in the AI industry. This groundbreaking suite is designed to provide a transparent, accessible, and supportive foundation for AI technology, offering developers the opportunity to utilize the StableLM base models for both commercial and research purposes under the CC BY-SA-4.0 license.

StableLM follows in the footsteps of Stability AI’s previous success with Stable Diffusion, an innovative image model that set a new standard for transparency and scalability in AI. The company’s latest release emphasizes its ongoing commitment to making AI technology available to all, showcasing the potential of small, efficient models when trained effectively.

Stability AI has built upon its experience in open-sourcing earlier language models, such as GPT-J, GPT-NeoX, and the Pythia suite, in collaboration with nonprofit research hub EleutherAI. These models were trained on The Pile, an open-source dataset that has laid the foundation for recent open-source language models like Cerebras-GPT and Dolly-2.

The Alpha version of StableLM, available in 3 billion and 7 billion parameters, boasts an impressive performance in conversational and coding tasks, thanks to its training on a new experimental dataset three times larger than The Pile, containing 1.5 trillion tokens. This dataset’s immense scale makes StableLM a formidable contender, even with its smaller parameter count compared to GPT-3’s 175 billion parameters.

Stability AI also plans to release research models that have been instruction fine-tuned, initially utilizing a combination of five recent open-source datasets for conversational agents, such as Alpaca, GPT4All, Dolly, ShareGPT, and HH. These fine-tuned models are designed for research purposes only, released under a noncommercial CC BY-NC-SA 4.0 license.

As language models become increasingly crucial in the digital economy, Stability AI aims to create AI technology that is transparent, accessible, and supportive. By open-sourcing their models, researchers can verify performance, explore interpretability techniques, and develop safeguards, while organizations can adapt these models for their own applications without relinquishing control or sharing sensitive data.

Stability AI’s focus on designing models for edge computing enables everyday users to run them on local devices, promoting widespread access to AI’s economic benefits. Furthermore, the company’s commitment to efficient and practical AI performance emphasizes their goal of developing tools that augment human creativity, productivity, and economic opportunity.

The new StableLM models are now available in Stability AI’s GitHub repository, with a full technical report expected soon. As the company continues to release new models and expand its team, it anticipates ongoing collaboration with developers and researchers, as well as community projects like Open Assistant and the crowd-sourced RLHF program to create open-source datasets for AI assistants.

https://stability.ai/blog/stability-ai-launches-the-first-of-its-stablelm-suite-of-language-models

https://github.com/stability-AI/stableLM/

{
"seed": 1649302223,
"used_random_seed": true,
"negative_prompt": "",
"num_outputs": 1,
"num_inference_steps": 300,
"guidance_scale": 7.5,
"width": 512,
"height": 512,
"vram_usage_level": "high",
"sampler_name": "euler",
"use_stable_diffusion_model": "revAnimated_v11",
"use_vae_model": "vae-ft-mse-840000-ema-pruned",
"stream_progress_updates": true,
"stream_image_progress": false,
"show_only_filtered_image": true,
"block_nsfw": false,
"output_format": "jpeg",
"output_quality": 75,
"output_lossless": false,
"metadata_output_format": "json",
"original_prompt": "Stability AI Debuts Revolutionary Open-Source Language Model Suite: StableLM, deepleaps.com",
"active_tags": [],
"inactive_tags": [],
"use_upscale": "RealESRGAN_x4plus",
"upscale_amount": "4",
"prompt": "Stability AI Debuts Revolutionary Open-Source Language Model Suite: StableLM, deepleaps.com",
"use_cpu": false
}