AI Research Skills (Orchestra)

A library of AI research skills that lets your agent fine-tune, train, evaluate and serve models, and take a project from idea to paper.

Pack · 98 skills

About

Ratings and reviews

What AI Research Skills (Orchestra) does

About

AI Research Skills is Orchestra Research's open-source library of skills for AI research and engineering. It gives your agent working knowledge of the frameworks researchers use, from fine-tuning and post-training to distributed training, inference and evaluation, plus a research layer that takes a project from literature survey and ideas through experiments to a written paper. What's in the pack - Autoresearch: runs a whole research project with an inner optimisation loop and an outer synthesis loop, routing to the other skills as it needs them - Ideas and papers: research brainstorming, creative thinking, ML paper writing with LaTeX templates and citation checks, academic plotting and conference talks - Training: model architectures (LitGPT, Mamba, RWKV, NanoGPT, TorchTitan), fine-tuning (Axolotl, LLaMA-Factory, Unsloth, PEFT), post-training (TRL, GRPO, OpenRLHF, SimPO, verl, slime, miles, torchforge) and distributed training (Megatron-Core, DeepSpeed, FSDP2, Accelerate, PyTorch Lightning, Ray Train) - Smaller and faster models: quantization (bitsandbytes, GPTQ, AWQ, HQQ, GGUF), Flash Attention, and serving with vLLM, TensorRT-LLM, llama.cpp and SGLang - Evaluation, safety and interpretability: lm-evaluation-harness, BigCode, NeMo Evaluator, LlamaGuard, NeMo Guardrails, Prompt Guard, Constitutional AI, TransformerLens, SAELens, pyvene and nnsight - Building with models: agents (LangChain, LlamaIndex, CrewAI, AutoGPT), RAG (Chroma, FAISS, Sentence Transformers, Pinecone, Qdrant), multimodal (CLIP, Whisper, LLaVA, Stable Diffusion, Segment Anything, BLIP-2, AudioCraft) and structured prompting (DSPy, Instructor, Guidance, Outlines) - Running experiments: tracking (Weights & Biases, MLflow, TensorBoard, SwanLab), observability (LangSmith, Phoenix) and GPU clouds (Modal, SkyPilot, Lambda Labs) - Agent-Native Research Artifacts: compile a paper, repo or notes into a structured research artifact, record how a project actually evolved, and review an artifact for rigor How to install - Run npx @orchestra-research/ai-research-skills: an interactive installer that finds your coding agents (Claude Code, Hermes Agent, OpenCode, Cursor, Gemini CLI and others) and installs everything, a quickstart bundle, a category or single skills - In Claude Code: /plugin marketplace add orchestra-research/AI-research-SKILLs, then install a category, such as /plugin install fine-tuning@ai-research-skills - Or ask your agent to read Orchestra's welcome doc and follow its install steps Good to know - Each skill pairs a short SKILL.md with reference docs drawn from the framework's official docs, real GitHub issues and release notes - The library is MIT licensed; the frameworks it covers have their own licenses - Orchestra Research also syncs these skills to its own research platform

Things to ask it

Once it’s added

“I need to fine-tune Llama 3 with custom data.”
“We need to scale training to 100 GPUs.”
“How do I optimize inference latency with vLLM?”
“Run GRPO reinforcement learning on this model with TRL.”
“Start an autoresearch project from this idea, through to a paper.”
“Write up these results as a NeurIPS paper with LaTeX.”

What it can do

Before you add it

Reads only

Each framework's bundled reference docs, known issues and release notes · Your code, data and experiment results

Makes changes · your agent can ask first

Training, evaluation and serving code and configs in your project · Papers, figures and research artifacts in your project

Skills in AI Research Skills (Orchestra)

Pack · 98 skills, each with its own page

academic-plottingSecurity scan passedGenerates publication-quality figures for ML papers from research context.
ara-compilerSecurity scan passedCompiles any research input — PDF papers, GitHub repositories, experiment logs, code directories, or raw notes — into a…
ara-research-managerSecurity scan passedRecords research provenance as a post-task epilogue, scanning conversation history at the end of a coding or research s…
ara-rigor-reviewerSecurity scan passedPerforms ARA Seal Level 2 semantic epistemic review on Agent-Native Research Artifacts, scoring six dimensions (evidenc…
audiocraft-audio-generationSecurity scan passedPyTorch library for audio generation including text-to-music (MusicGen) and text-to-sound (AudioGen).
autogpt-agentsSecurity scan passedAutonomous AI agent platform for building and deploying continuous agents.
autoresearchSecurity scan passedOrchestrates end-to-end autonomous AI research projects using a two-loop architecture.
awq-quantizationSecurity scan passedActivation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss.
axolotlSecurity scan passedExpert guidance for fine-tuning LLMs with Axolotl - YAML configs, 100+ models, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, multimoda…
blip-2-vision-languageSecurity scan passedVision-language pre-training framework bridging frozen image encoders and LLMs.
brainstorming-research-ideasSecurity scan passedGuides researchers through structured ideation frameworks to discover high-impact research directions.
chromaSecurity scan passedOpen-source embedding database for AI applications.
clipSecurity scan passedOpenAI's model connecting vision and language.
constitutional-aiSecurity scan passedAnthropic's method for training harmless AI through self-improvement.
creative-thinking-for-researchSecurity scan passedApplies cognitive science frameworks for creative thinking to CS and AI research ideation.
crewai-multi-agentSecurity scan passedMulti-agent orchestration framework for autonomous AI collaboration.
deepspeedSecurity scan passedExpert guidance for distributed training with DeepSpeed - ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8…
distributed-llm-pretraining-torchtitanSecurity scan passedProvides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP).
dspySecurity scan passedBuild complex AI systems with declarative programming, optimize prompts automatically, create modular RAG systems and a…
evaluating-code-modelsSecurity scan passedEvaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics.
evaluating-cosmos-policySecurity scan passedEvaluates NVIDIA Cosmos Policy on LIBERO and RoboCasa simulation environments.
evaluating-llms-harnessSecurity scan passedEvaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag).
evolving-ai-agentsSecurity scan passedProvides guidance for automatically evolving and optimizing AI agents across any domain using LLM-driven evolution algo…
experiment-tracking-swanlabSecurity scan passedProvides guidance for experiment tracking with SwanLab.
faissSecurity scan passedFacebook's library for efficient similarity search and clustering of dense vectors.
fine-tuning-openvla-oftSecurity scan passedFine-tunes and evaluates OpenVLA-OFT and OpenVLA-OFT+ policies for robot action generation with continuous action heads…
fine-tuning-serving-openpiSecurity scan passedFine-tune and serve Physical Intelligence OpenPI models (pi0, pi0-fast, pi0.5) using JAX or PyTorch backends for robot…
fine-tuning-with-trlSecurity scan passedFine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GR…
gguf-quantizationSecurity scan passedGGUF format and llama.cpp quantization for efficient CPU/GPU inference.
gptqSecurity scan passedPost-training 4-bit quantization for LLMs with minimal accuracy loss.
grpo-rl-trainingSecurity scan passedExpert guidance for GRPO/RL fine-tuning with TRL for reasoning and task-specific model training
guidanceSecurity scan passedControl LLM output with regex and grammars, guarantee valid JSON/XML/code generation, enforce structured formats, and b…
hqq-quantizationSecurity scan passedHalf-Quadratic Quantization for LLMs without calibration data.
huggingface-accelerateSecurity scan passedSimplest distributed training API.
huggingface-tokenizersSecurity scan passedFast tokenizers optimized for research and production.
implementing-llms-litgptSecurity scan passedImplements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mist…
instructorSecurity scan passedExtract structured data from LLM responses with Pydantic validation, retry failed extractions automatically, parse comp…
knowledge-distillationSecurity scan passedCompress large language models using knowledge distillation from teacher to student models.
lambda-labs-gpu-cloudSecurity scan passedReserved and on-demand GPU cloud instances for ML training and inference.
langchainSecurity scan passedFramework for building LLM-powered applications with agents, chains, and RAG.
langsmith-observabilitySecurity scan passedLLM observability platform for tracing, evaluation, and monitoring.
llama-cppSecurity scan passedRuns LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware.
llama-factorySecurity scan passedExpert guidance for fine-tuning LLMs with LLaMA-Factory - WebUI no-code, 100+ models, 2/3/4/5/6/8-bit QLoRA, multimodal…
llamaguardSecurity scan passedMeta's 7-8B specialized moderation model for LLM input/output filtering.
llamaindexSecurity scan passedData framework for building LLM applications with RAG.
llavaSecurity scan passedLarge Language and Vision Assistant.
long-contextSecurity scan passedExtend context windows of transformer models using RoPE, YaRN, ALiBi, and position interpolation techniques.
mamba-architectureSecurity scan passedState-space model with O(n) complexity vs Transformers' O(n²).
miles-rl-trainingSecurity scan passedProvides guidance for enterprise-grade RL training using miles, a production-ready fork of slime.
mlflowSecurity scan passedTrack ML experiments, manage model registry with versioning, deploy models to production, and reproduce experiments wit…
ml-paper-writingSecurity scan passedWrite publication-ready ML/AI papers for NeurIPS, ICML, ICLR, ACL, AAAI, COLM.
ml-training-recipesSecurity scan passedBattle-tested PyTorch training recipes for all domains — LLMs, vision, diffusion, medical imaging, protein/drug discove…
modal-serverless-gpuSecurity scan passedServerless GPU cloud platform for running ML workloads.
model-mergingSecurity scan passedMerge multiple fine-tuned models using mergekit to combine capabilities without retraining.
model-pruningSecurity scan passedReduce LLM size and accelerate inference using pruning techniques like Wanda and SparseGPT.
moe-trainingSecurity scan passedTrain Mixture of Experts (MoE) models using DeepSpeed or HuggingFace.
nanogptSecurity scan passedEducational GPT implementation in ~300 lines.
nemo-curatorSecurity scan passedGPU-accelerated data curation for LLM training.
nemo-evaluator-sdkSecurity scan passedEvaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execu…
nemo-guardrailsSecurity scan passedNVIDIA's runtime safety framework for LLM applications.
nnsight-remote-interpretabilitySecurity scan passedProvides guidance for interpreting and manipulating neural network internals using nnsight with optional NDIF remote ex…
openrlhf-trainingSecurity scan passedHigh-performance RLHF framework with Ray+vLLM acceleration.
optimizing-attention-flashSecurity scan passedOptimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction.
outlinesSecurity scan passedGuarantee valid JSON/XML/code structure during generation, use Pydantic models for type-safe outputs, support local mod…
peft-fine-tuningSecurity scan passedParameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods.
phoenix-observabilitySecurity scan passedOpen-source AI observability platform for LLM tracing, evaluation, and monitoring.
pineconeSecurity scan passedManaged vector database for production AI applications.
presenting-conference-talksSecurity scan passedGenerates conference presentation slides (Beamer LaTeX PDF and editable PPTX) from a compiled paper with speaker notes…
prompt-guardSecurity scan passedMeta's 86M prompt injection and jailbreak detector.
pytorch-fsdp2Security scan passedAdds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and d…
pytorch-lightningSecurity scan passedHigh-level PyTorch framework with Trainer class, automatic distributed training (DDP/FSDP/DeepSpeed), callbacks system,…
pyvene-interventionsSecurity scan passedProvides guidance for performing causal interventions on PyTorch models using pyvene's declarative intervention framewo…
qdrant-vector-searchSecurity scan passedHigh-performance vector similarity search engine for RAG and semantic search.
quantizing-models-bitsandbytesSecurity scan passedQuantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss.
ray-dataSecurity scan passedScalable data processing for ML workloads.
ray-trainSecurity scan passedDistributed training orchestration across clusters.
rwkv-architectureSecurity scan passedRNN+Transformer hybrid with O(n) inference.
segment-anything-modelSecurity scan passedFoundation model for image segmentation with zero-shot transfer.
sentencepieceSecurity scan passedLanguage-independent tokenizer treating text as raw Unicode.
sentence-transformersSecurity scan passedFramework for state-of-the-art sentence, text, and image embeddings.
serving-llms-vllmSecurity scan passedServes LLMs with high throughput using vLLM's PagedAttention and continuous batching.
sglangSecurity scan passedFast structured generation and serving for LLMs with RadixAttention prefix caching.
simpo-trainingSecurity scan passedSimple Preference Optimization for LLM alignment.
skypilot-multi-cloud-orchestrationSecurity scan passedMulti-cloud orchestration for ML workloads with automatic cost optimization.
slime-rl-trainingSecurity scan passedProvides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework.
sparse-autoencoder-trainingSecurity scan passedProvides guidance for training and analyzing Sparse Autoencoders (SAEs) using SAELens to decompose neural network activ…
speculative-decodingSecurity scan passedAccelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques.
stable-diffusion-image-generationSecurity scan passedState-of-the-art text-to-image generation with Stable Diffusion models via HuggingFace Diffusers.
systems-paper-writingSecurity scan passedComprehensive guide for writing systems papers targeting OSDI, SOSP, ASPLOS, NSDI, and EuroSys.
tensorboardSecurity scan passedVisualize training metrics, debug models with histograms, compare experiments, visualize model graphs, and profile perf…
tensorrt-llmSecurity scan passedOptimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency.
torchforge-rl-trainingSecurity scan passedProvides guidance for PyTorch-native agentic RL using torchforge, Meta's library separating infra from algorithms.
training-llms-megatronSecurity scan passedTrains large language models (2B-462B parameters) using NVIDIA Megatron-Core with advanced parallelism strategies.
transformer-lens-interpretabilitySecurity scan passedProvides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer…
unslothSecurity scan passedExpert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization
verl-rl-trainingSecurity scan passedProvides guidance for training LLMs with reinforcement learning using verl (Volcano Engine RL).
weights-and-biasesSecurity scan passedTrack ML experiments with automatic logging, visualize training in real-time, optimize hyperparameters with sweeps, and…
whisperSecurity scan passedOpenAI's general-purpose speech recognition model.

Install AI Research Skills (Orchestra)

Your agent

Runs on your machine.

In ChatGPTDownload the AI-Research-SKILLs folder and zip it, then Plugins → Skills → Create → Upload from your computer. Business and Enterprise plans.

View the skills on GitHub

Runs on your machine.