LLM resources
109 vendor-neutral picks: model APIs, local runners, serving engines, frameworks, vector databases, evaluation tools, courses and the papers that started it all.
Model APIs & platforms
Hosted developer platforms and gateways where you send prompts to a model and pay per token, from first-party labs to multi-provider aggregators and cloud marketplaces.
- OpenAI APIOpenAI's developer documentation, with quickstarts, the Responses API, a model overview and guides for text, vision, audio and agents. Start here when you want to call GPT models or build with the Agents SDK.
- Claude Platform (Anthropic API)Anthropic's developer documentation, from first API call to production use, covering the Messages API, the Claude model family and agent building blocks. Use it when you plan to integrate Claude into an application.
- Google AI Studio & Gemini APIGoogle's developer docs for the Gemini API, with AI Studio for trying prompts and generating API keys. A quick route to multimodal Gemini models when you want to prototype before moving to Google Cloud.
- Gemini Enterprise Agent Platform (formerly Vertex AI)Google Cloud's managed platform for building, governing and deploying AI agents and apps with Gemini plus partner and open models. Old Vertex AI documentation links now redirect here; choose it for enterprise workloads on Google Cloud.
- Mistral Studio & APIDocumentation for Mistral's developer console and API, where you call its hosted models and manage keys, usage and administration. A sensible pick if you want a European provider that also releases open-weight models.
- Cohere APICohere's API documentation covering text generation, embeddings and reranking. A practical starting point when your project centers on enterprise search, retrieval-augmented generation or document classification rather than open-ended chat.
- Grok API (xAI)Developer documentation for the Grok API, covering text, code, voice, image and video models, plus tools, files and deployment guides. Use it when you want to call Grok models from your own applications.
- DeepSeek APIDocs for DeepSeek's hosted API, which accepts OpenAI-style and Anthropic-style request formats, so existing client code often needs only a new base URL and key. Use it to try DeepSeek models from tools you already run.
- OpenRouterOne API and billing account that routes requests to hundreds of models from many providers. Useful for comparing models quickly, adding fallbacks, or avoiding separate contracts with every lab.
- Amazon BedrockAWS's managed service for calling foundation models from several providers through one API inside your AWS account. Choose it when security reviews, IAM controls and existing AWS billing drive your vendor decisions.
- Microsoft Foundry (formerly Azure AI Foundry)Microsoft's platform for building AI agents, models and apps on Azure, renamed from Azure AI Foundry. Use it when your organization standardizes on Microsoft cloud services and wants hosted models with enterprise governance.
Model hubs & leaderboards
Places to find open model weights and to compare models on benchmarks, human votes, price and speed before you commit to one.
- Hugging Face HubThe main public registry for open models, datasets and demo apps, hosting millions of repositories with model cards and version history. Search it when you need open weights, a dataset or a reference implementation.
- Arena (formerly LMArena)A public leaderboard built from anonymous side-by-side votes, where people compare two models' answers without knowing which is which. Treat it as one signal of conversational quality, alongside task-specific benchmarks.
- Artificial AnalysisIndependent comparisons of AI models and API providers on intelligence scores, price, output speed and latency. Handy when you are choosing both a model and a host and need quality and cost side by side.
- LiveBenchA benchmark built to resist test-set contamination, using questions refreshed over time and scored automatically against ground-truth answers. Check it for objective comparisons of reasoning, coding and math ability across models.
- Epoch AI Benchmarking HubEpoch AI's database of benchmark results, mixing evaluations the team runs itself with figures from outside sources. Use it to see how leading models score on hard tasks and to compare them across several tests.
- SWE-benchA benchmark of real GitHub issues from Python repositories, with public leaderboards for variants such as Verified, Multilingual and Lite. Use it to judge how well a model or coding agent resolves genuine software tasks.
- Stanford HELMStanford's Holistic Evaluation of Language Models, an open framework and living set of leaderboards that score models across many scenarios and metrics rather than accuracy alone. Use it for transparent, reproducible comparisons.
Run models locally
Apps and runtimes that let you download open-weight models and run them on your own laptop, workstation or server.
- OllamaA runtime that pulls open models with one command and serves them through a local REST API on macOS, Windows and Linux. Choose it for quick experiments and as the backend for other local tools; hosted cloud models are optional.
- LM StudioA desktop app for discovering, downloading and chatting with local models, built on llama.cpp and MLX runtimes and able to expose a local server. Pick it if you want a graphical interface rather than a command line.
- llama.cppA C/C++ inference engine behind much of the local-model ecosystem, with GGUF quantization and backends for CPUs, Apple silicon and GPUs. Use it directly for tight control over memory, or through the apps built on it.
- MLX LMA Python package for running and fine-tuning language models on Apple silicon with the MLX framework. Reach for it when a Mac is your main machine and you want quantized models and LoRA training in Python.
- JanA free desktop chat app that runs open models on your own computer and can also connect to hosted services such as OpenAI or Anthropic. Suited to people who want a private, ChatGPT-style window they control.
- Open WebUIA self-hosted chat interface for Ollama and OpenAI-compatible endpoints that you can extend with Python. Its license requires keeping the Open WebUI branding, so read the terms before white-labeling it.
Serving & inference engines
Open-source inference engines and hosted GPU platforms for serving models at production speed, from self-managed clusters to pay-per-token endpoints.
- vLLMA widely used open-source engine for high-throughput LLM serving, built around continuous batching and efficient KV-cache memory management. Choose it to host open models behind an OpenAI-compatible API on your own GPUs.
- SGLangA high-performance serving framework for language and multimodal models, with strong support for structured output and large mixture-of-experts models. Worth benchmarking against vLLM when you run big open models on your own GPUs.
- TensorRT-LLMNVIDIA's library for optimized LLM inference on its GPUs, with a Python API, quantization, speculative decoding, LoRA support and a trtllm-serve command. Consider it when you run NVIDIA hardware at scale and want vendor-tuned performance.
- Together AIA cloud for running open models through serverless or dedicated endpoints, with managed fine-tuning and GPU compute under the same roof. A fit when you want open-weight models without operating inference infrastructure yourself.
- Fireworks AIA platform for serving open models and training specialized versions of them, with fine-tuning offered alongside inference. Consider it when you plan to tune a model on your own data and deploy it on the same service.
- Groq (GroqCloud)A hosted inference service running open models on Groq's custom chips, with an OpenAI-compatible API. Try it when response speed matters most, such as voice agents and interactive tools.
- Cerebras InferenceInference served from Cerebras's wafer-scale chips through an OpenAI-compatible API, with a free trial for developers. Look at it when you need very fast generation from the open models it supports.
- DeepInfraA pay-as-you-go API for running many open models without managing GPUs, with usage-based pricing. A reasonable place to price-check open-weight models against larger providers before committing.
- ReplicateA hosted service for running open and custom models through a simple API, billed by usage. Useful for trying community models, including image and audio ones, or packaging your own model for deployment.
- ModalServerless cloud infrastructure where you define GPU workloads in Python and scale them to zero when idle. Use it to deploy your own inference servers, batch jobs or fine-tuning runs without managing clusters.
- RunpodA GPU cloud that rents pods by the hour or second and offers serverless endpoints that scale to zero. A practical option for self-hosting an open model or running a fine-tune on a budget.
Frameworks & orchestration
Libraries that wire models, prompts, tools and data together so you can build chains, pipelines and applications without hand-rolling every call.
- LangChainA widely adopted open-source toolkit with standard interfaces for models, tools and retrievers, available in Python and TypeScript. Start here for prebuilt integrations, and add LangGraph when you need stateful agent workflows.
- LangGraphA low-level orchestration framework that models agents as graphs, with persistent state and human-in-the-loop checkpoints. Choose it when a simple chain is not enough and you need explicit control over loops and branching.
- LlamaIndexA framework and hosted services for turning documents into agent-ready context, including parsing, extraction and indexing. Use it when your main challenge is getting messy PDFs and files into a reliable retrieval pipeline.
- Haystackdeepset's open-source Python framework for building search, RAG and agent pipelines from modular components you can inspect and debug. A good fit for teams that want explicit, testable pipelines in production.
- DSPyA Python framework from Stanford NLP that replaces hand-written prompts with declarative signatures and modules, then optimizes them against your own metric. Reach for it when prompts keep breaking as models or tasks change.
- Semantic KernelMicrosoft's open-source SDK for building AI agents in C#, Python and Java. Microsoft now names Agent Framework as its successor, so consider Semantic Kernel mainly when maintaining existing code.
- Vercel AI SDKA TypeScript toolkit for building AI apps and agents with one interface across model providers, plus streaming helpers and UI hooks for frameworks like React. A natural choice for Next.js and other JavaScript front ends.
- Pydantic AIA typed Python agent framework from the Pydantic team, in which tools, dependencies and outputs are validated through type hints. Choose it for production agents when you want structured results and strong editor support.
Vector databases & retrieval
Databases, embedding models and rerankers that power semantic search and retrieval-augmented generation over your own documents.
- pgvectorAn extension that adds vector columns and similarity search to PostgreSQL, with exact search and approximate indexes. The simplest route to retrieval when your application data already lives in Postgres.
- PineconeA fully managed vector database service aimed at production search and retrieval without cluster operations. Consider it when you want a hosted index and would rather not run database infrastructure yourself.
- WeaviateAn open-source AI database that combines vector search with keyword filtering and built-in options for embeddings and retrieval-augmented generation. Run it yourself or use its managed cloud when you need hybrid search at scale.
- QdrantAn open-source vector search engine written in Rust, with payload filtering and a managed cloud option. A solid candidate when you want fast, filtered similarity search that you can also self-host.
- MilvusA cloud-native, open-source vector database built to handle very large collections across distributed clusters. Look at it for large-scale similarity search; Zilliz Cloud offers it as a managed service.
- ChromaAn Apache 2.0 search backend for AI applications, covering vector, full-text, regex and metadata queries. Easy to start with locally, which makes it popular for prototypes that may later grow into production.
- Elasticsearch vector searchElastic's search engine doubles as a vector database, so one system can combine keyword relevance, filters and embedding similarity. A natural pick if you already operate Elasticsearch for search or log analytics.
- MTEB LeaderboardThe Massive Text Embedding Benchmark leaderboard on Hugging Face, ranking embedding models on retrieval, classification, clustering and multilingual tasks. Filter it by language to shortlist models before testing them on your own data.
- Sentence TransformersA Python library for computing text embeddings and for training embedding, reranker and sparse retrieval models, with many pretrained checkpoints on Hugging Face. Use it to run open embedding models locally or fine-tune one on your domain.
Evaluation & observability
Tools for testing prompts and models, scoring outputs, and tracing what your LLM application actually does in production.
- promptfooRuns prompt tests and red-team scans against many models from declarative configs, in the terminal or CI. OpenAI acquired the company in 2026; its README says the project remains open source under the MIT license.
- RagasA Python library for evaluating LLM applications, best known for retrieval-augmented generation metrics such as faithfulness and context precision. Use it to replace informal spot checks with repeatable scores and test sets.
- DeepEvalAn Apache 2.0 framework that brings unit-test style evaluation to LLM apps, RAG pipelines and agents, with many built-in metrics. A good fit for teams that want evals to run inside pytest and CI.
- LangfuseAn MIT-licensed platform for tracing, prompt management and evaluation of LLM applications, self-hostable or available as a cloud service. It joined ClickHouse in January 2026, and the product remains open source.
- LangSmithLangChain's hosted platform for tracing, monitoring and evaluating agents and LLM apps, and it works with many frameworks beyond LangChain. Consider it if you already use LangChain or LangGraph and want first-party tooling.
- Arize PhoenixA self-hostable tool for tracing LLM calls, running evaluations and comparing prompt experiments. Arize also sells a separate managed platform, so review Phoenix's license terms before embedding it in a product.
- BraintrustA commercial platform that pairs production tracing with evals, so teams can score model output, compare experiments and catch regressions before release. Evaluate it if you want a managed workflow instead of assembling open-source parts.
- InspectAn MIT-licensed evaluation framework from the UK AI Security Institute for building benchmarks and agent evaluations, with tool use, multi-turn dialog and model-graded scoring. Use it for rigorous, reproducible model testing.
Fine-tuning & training
Libraries and managed services for adapting open models to your data with supervised fine-tuning, LoRA, and preference or reinforcement learning methods.
- Hugging Face TRLHugging Face's library for post-training transformer models with supervised fine-tuning, preference optimization such as DPO, and reinforcement learning methods. A standard starting point for aligning an open model on your own data.
- Hugging Face PEFTA library for parameter-efficient fine-tuning that trains small adapter weights, such as LoRA, instead of every parameter. Use it to tune large models on modest GPUs and keep each variant as a small file.
- UnslothA fine-tuning and reinforcement learning toolkit built to cut memory use and training time on a single GPU, now with a local app for running and training models. Suited to individuals and small teams tuning open models.
- AxolotlAn open-source tool for fine-tuning and post-training language models from a YAML configuration, covering techniques from LoRA to multi-GPU runs. Pick it for repeatable, config-driven training without writing your own training loop.
- LLaMA-FactoryAn Apache 2.0 toolkit for fine-tuning more than 100 language and vision-language models, with a web interface and support for LoRA, QLoRA and preference tuning. Handy if you want training without writing much code.
- Fireworks Training APIFireworks's managed training service for supervised fine-tuning, preference optimization and reinforcement learning on open models, with serverless and dedicated options. Consider it when you want to tune and then deploy on one platform.
- Together AI Fine-TuningTogether AI's managed fine-tuning for open models, accepting instruction, preference and plain-text datasets and serving the result from dedicated endpoints. Useful when you want to tune a model without provisioning GPUs yourself.
Agents & MCP
Protocols and SDKs for building tool-using agents, connecting models to external systems and coordinating several agents.
- Model Context ProtocolAn open standard for connecting AI applications to external data, tools and workflows through a common client-server protocol. Read it when you want assistants to use your own systems or to reuse existing MCP servers.
- Agent2Agent (A2A) ProtocolAn open protocol, now hosted by the Linux Foundation, that lets agents built on different frameworks discover each other's capabilities and collaborate on tasks. Relevant when agents from separate teams or vendors must interoperate.
- OpenAI Agents SDKA lightweight Python framework for multi-agent workflows with handoffs, guardrails and tracing, positioned as the production-ready successor to OpenAI's experimental Swarm project. Start here for agents built around OpenAI's APIs.
- Claude Agent SDKAnthropic's SDK for building agents on the same loop, tools and context management that power Claude Code, available for Python and TypeScript. Use it for coding, research or file-handling agents.
- Google Agent Development KitGoogle's open-source framework for building, debugging and deploying agents, with SDKs for Python, TypeScript, Go, Java and Kotlin and support for graph-based workflows. Model-agnostic by design, though tuned for Gemini and Google Cloud.
- Microsoft Agent FrameworkMicrosoft's framework for agents and graph-based multi-agent workflows in .NET and Python, with Go in public preview. It is the direct successor to Semantic Kernel and AutoGen, and migration guides cover both.
- CrewAIA Python framework for defining role-based agents, called crews, and event-driven flows that coordinate them. Reach for it when a task splits naturally into specialist roles and you want a readable, high-level setup.
- Strands AgentsAn open-source SDK from AWS for building production agents in Python and TypeScript, designed to work with models from many providers on any cloud. A lightweight choice for teams already building on AWS.
Safety & security
Risk frameworks, guardrail libraries and red-teaming tools for finding and reducing prompt injection, data leakage and other LLM-specific failures.
- OWASP Top 10 for LLM ApplicationsOWASP's ranked list of the most critical risks in LLM applications, led in the 2025 edition by prompt injection, sensitive information disclosure and supply chain weaknesses. Use it as a threat-modeling checklist before launch.
- NIST AI Risk Management FrameworkNIST's voluntary framework for identifying and managing AI risks, with a companion Generative AI Profile (AI 600-1). NIST says AI RMF 1.0 is being revised, so check the page for the current version.
- MITRE ATLASA knowledge base of adversary tactics, techniques and case studies for attacks on AI-enabled systems, organized like MITRE ATT&CK. Use it to map threats and structure red-team exercises in a shared vocabulary.
- NVIDIA NeMo GuardrailsAn open-source toolkit for adding programmable guardrails to LLM-based conversational systems, steering topics and filtering inputs and outputs. Consider it when you want policy defined in configuration rather than scattered through prompts.
- Guardrails AIA Python framework that wraps LLM calls with validators for format, safety and content rules, and can re-ask or fix output that fails. A good fit for enforcing structured, policy-compliant responses.
- garakNVIDIA's open-source LLM vulnerability scanner, which probes a model for prompt injection, data leakage, jailbreaks and other failures using a library of attack probes. Run it before release, much as you would scan a server.
- PyRITMicrosoft's open-source Python Risk Identification Tool for generative AI, which automates red-team campaigns against models and applications. Use it to scale adversarial testing beyond manual probing.
Courses & learning
Free and low-cost courses, lecture series and code-along projects that take you from transformer basics to building and training language models.
- DeepLearning.AI Short CoursesShort, hands-on courses taught with industry partners on prompting, RAG, agents, fine-tuning and evaluation, each with runnable code. Good for learning one technique in an evening without committing to a full curriculum.
- Hugging Face LearnFree, code-first courses from Hugging Face, including a course on Transformers and LLMs, an Agents course and an MCP course, plus cookbook recipes. Work through them to learn the open-model ecosystem by building.
- fast.ai Practical Deep LearningJeremy Howard's free, code-first course that builds working deep-learning models first and explains the theory afterward. A strong foundation in training loops, fine-tuning and NLP before you move on to LLM-specific material.
- Neural Networks: Zero to HeroAndrej Karpathy's video series that builds neural networks in code from the basics up to a GPT-style transformer, with notebooks in the linked repository. Watch it to understand what happens inside a language model.
- Stanford CS336: Language Modeling from ScratchStanford's course on building a language model end to end, with assignments on tokenization, training, systems efficiency, scaling laws, data and alignment. Public course materials let you follow along on your own.
- Prompt Engineering GuideAn open, community-maintained guide to prompting techniques, with explanations, examples and paper links covering chain-of-thought, RAG, agents and more. Use it as a reference when designing or debugging prompts.
- LLMs from Scratch (Sebastian Raschka)Sebastian Raschka's companion repository for building a GPT-style model step by step in PyTorch, covering tokenization, attention, pretraining and instruction fine-tuning in notebooks. Ideal if you learn best by reading and running code.
Key papers
The research papers behind today's models, from the original transformer to scaling laws, alignment methods and open reasoning models, linked to arXiv.
- Attention Is All You Need (Vaswani et al., 2017)The paper that introduced the transformer, replacing recurrence with self-attention so models train in parallel and scale well. Read it for the architecture that nearly every modern LLM builds on.
- Language Models Are Few-Shot Learners (Brown et al., 2020)OpenAI's GPT-3 paper, showing that a 175-billion-parameter model can pick up new tasks from a few examples in the prompt, with no weight updates. It established in-context learning as a core idea.
- Training Language Models to Follow Instructions with Human Feedback (Ouyang et al., 2022)OpenAI's InstructGPT paper, which tunes GPT-3 with human demonstrations and preference-based reinforcement learning. Raters preferred a 1.3B tuned model over the 175B base model, making RLHF the template for chat assistants.
- Training Compute-Optimal Large Language Models (Hoffmann et al., 2022)DeepMind's Chinchilla study, which found that model size and training tokens should grow together, and showed a 70B model trained on far more data beating much larger ones. Use it to reason about compute-optimal training budgets.
- LLaMA: Open and Efficient Foundation Language Models (Touvron et al., 2023)Meta's paper introducing the LLaMA models, trained on publicly available data at sizes from 7B to 65B parameters, with the smaller ones competitive with far larger models. It helped spark the open-weight ecosystem of fine-tuned derivatives.
- Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (Wei et al., 2022)Google researchers show that including worked reasoning steps in the prompt substantially improves large models on arithmetic and logic tasks. The idea behind step-by-step prompting and a precursor to dedicated reasoning models.
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020)The paper that named retrieval-augmented generation, pairing a text generator with a retriever over a document index so answers draw on external knowledge. Read it for the origin of the RAG pattern.
- LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., 2021)Introduces training small low-rank update matrices while the base weights stay frozen, sharply cutting trainable parameters and memory. The technique behind most adapter-based fine-tuning of open models.
- QLoRA: Efficient Finetuning of Quantized LLMs (Dettmers et al., 2023)Shows how to fine-tune a 4-bit quantized model through LoRA adapters, making it possible to tune a 65B-parameter model on a single 48 GB GPU. Essential background for budget fine-tuning.
- Direct Preference Optimization: Your Language Model Is Secretly a Reward Model (Rafailov et al., 2023)Derives a simple classification-style loss that optimizes a model directly from preference pairs, skipping the separate reward model and reinforcement learning loop of RLHF. A common choice for aligning open models.
- Constitutional AI: Harmlessness from AI Feedback (Bai et al., 2022)Anthropic's method for training a harmless assistant from a written set of principles, using AI-generated critiques and preference labels instead of human harm labels. It introduced reinforcement learning from AI feedback.
- Mixtral of Experts (Jiang et al., 2024)Mistral's sparse mixture-of-experts model, which routes each token to two of eight expert networks per layer, so only a fraction of its parameters is active per token. A readable case study in MoE efficiency.
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (DeepSeek-AI, 2025)DeepSeek's report on training reasoning models with large-scale reinforcement learning, including an R1-Zero variant trained without a supervised fine-tuning stage, and on distilling the results into smaller models. Key reading on open reasoning models.
- Scaling Laws for Neural Language Models (Kaplan et al., 2020)OpenAI's study showing that language-model loss falls as a smooth power law with model size, dataset size and compute. It set up the scaling debate that Chinchilla later refined.
Communities & newsletters
Forums, newsletters and blogs where practitioners share releases, benchmarks and hard-won lessons about building with language models.
- r/LocalLLaMAA busy subreddit where people trade local-model benchmarks, quantization results, hardware builds and first impressions of new open-weight releases. Useful for practical reports on what actually runs well on consumer machines.
- Hugging Face ForumsThe official Hugging Face discussion forum for questions about Transformers, datasets, training, Spaces and inference. A reasonable place to ask for help with open-model tooling and to see how others solved similar problems.
- Latent SpaceA newsletter and podcast for AI engineers, featuring interviews with builders, analysis of model releases and deep dives into tooling. Follow it to keep up with how practitioners ship LLM products.
- Simon Willison's WeblogIndependent developer Simon Willison's running notes on new models, tools and techniques, with frequent hands-on tests and clear writing about prompt injection. A dependable daily read for developers building with LLMs.
- InterconnectsNathan Lambert's newsletter on open models, post-training and the AI industry, written from a researcher's vantage point. Worth reading for context on reinforcement learning, open releases and where the field is heading.
- Hugging Face Daily PapersA daily, community-voted list of notable new arXiv papers, with discussion threads and links to related models and demos. Skim it to see which research practitioners are paying attention to.
Know a tool that belongs here? Suggest it. Editorial picks are never paid for; sponsored listings are labeled.
Need hands-on help building it?
RAG, agents, fine-tuning or a production chatbot: describe the project and we'll suggest an approach and, if you want, partners who build it.
- Free and vendor-neutral: no model provider pays for placement.
- Your details go only to partners you agree to.