Skip to content

@deepinfra Deep Infra

Change the repository type filter

All

    Repositories list

    42 repositories

    • TensorRT-LLM

      Public
      TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optim…
      Python
      •
      Other
      •2.3k•0•0•1•Updated Apr 23, 2026Apr 23, 2026

    • docs

      Public
      DeepInfra platform documentation
      MDX
      •
      MIT License
      •0•0•0•0•Updated Apr 21, 2026Apr 21, 2026

    • dynamo

      Public
      A Datacenter Scale Distributed Inference Serving Framework
      Rust
      •
      Other
      •1k•0•0•2•Updated Apr 17, 2026Apr 17, 2026

    • hub-docs

      Public
      Docs of the Hugging Face Hub
      Handlebars
      •
      Apache License 2.0
      •438•0•0•0•Updated Apr 14, 2026Apr 14, 2026

    • vllm-omni

      Public
      A framework for efficient model inference with omni-modality models
      Python
      •
      Apache License 2.0
      •819•0•0•0•Updated Mar 6, 2026Mar 6, 2026

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •16k•0•0•1•Updated Feb 27, 2026Feb 27, 2026

    • huggingface.js

      Public
      Use Hugging Face with JavaScript
      TypeScript
      •
      MIT License
      •682•0•0•0•Updated Feb 23, 2026Feb 23, 2026

    • tiktoken

      Public
      tiktoken is a fast BPE tokeniser for use with OpenAI's models.
      Python
      •
      MIT License
      •1.4k•0•0•0•Updated Feb 8, 2026Feb 8, 2026

    • Model-Optimizer

      Public
      A unified library of SOTA model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models …
      Python
      •
      Apache License 2.0
      •366•0•0•0•Updated Jan 12, 2026Jan 12, 2026

    • cookbooks

      Public
      A collection of cookbooks, tutorials, and examples for using AI models on DeepInfra. This repository provides practical guides, performance benchmarks, and prod…
      Jupyter Notebook
      •0•0•0•0•Updated Dec 15, 2025Dec 15, 2025

    • openbench

      Public
      Provider-agnostic, open-source evaluation infrastructure for language models
      Python
      •
      MIT License
      •98•0•0•0•Updated Nov 13, 2025Nov 13, 2025

    • sglang

      Public
      SGLang is a fast serving framework for large language models and vision language models.
      Python
      •
      Apache License 2.0
      •5.5k•0•0•0•Updated Oct 14, 2025Oct 14, 2025

    • SpecForge

      Public
      Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
      Python
      •
      MIT License
      •213•0•0•0•Updated Oct 8, 2025Oct 8, 2025

    • Roo-Code

      Public
      Roo Code gives you a whole dev team of AI agents in your code editor.
      TypeScript
      •
      Apache License 2.0
      •3.1k•0•0•0•Updated Sep 4, 2025Sep 4, 2025

    • kilocode

      Public
      Open Source AI coding assistant for planning, building, and fixing code. We're a superset of Roo, Cline, and our own features. Follow us: kilocode.ai/social
      TypeScript
      •
      Apache License 2.0
      •2.4k•0•0•0•Updated Aug 28, 2025Aug 28, 2025

    • ocr-tools

      Public
      Python
      •2•5•1•0•Updated Aug 2, 2025Aug 2, 2025

    • transformers

      Public
      🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
      Python
      •
      Apache License 2.0
      •33k•0•0•0•Updated Jul 29, 2025Jul 29, 2025

    • olmocr

      Public
      Toolkit for linearizing PDFs for LLM datasets/training
      Python
      •
      Apache License 2.0
      •1.4k•0•0•0•Updated Jul 9, 2025Jul 9, 2025

    • Kokoro-FastAPI

      Public
      Dockerized FastAPI wrapper for Kokoro-82M text-to-speech model w/CPU ONNX and NVIDIA GPU PyTorch support, handling, and auto-stitching
      Python
      •
      Apache License 2.0
      •788•0•0•0•Updated May 28, 2025May 28, 2025

    • tensorrtllm_backend

      Public
      The Triton TensorRT-LLM Backend
      Python
      •
      Apache License 2.0
      •138•0•0•0•Updated May 8, 2025May 8, 2025

    • deepinfra-chat

      Public
      Sample Next.js ai chat app using Deep Infra inference and Vercel ai sdk
      TypeScript
      •2•1•0•0•Updated Mar 17, 2025Mar 17, 2025

    • cutlass

      Public
      CUDA Templates for Linear Algebra Subroutines
      C++
      •
      Other
      •1.8k•0•0•0•Updated Mar 15, 2025Mar 15, 2025

    • flash-attention

      Public
      Fast and memory-efficient exact attention
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •2.6k•0•0•0•Updated Feb 20, 2025Feb 20, 2025

    • Zonos

      Public
      Python
      •
      Apache License 2.0
      •815•0•0•0•Updated Feb 12, 2025Feb 12, 2025

    • Pyramid-Flow

      Public
      Code of Pyramidal Flow Matching for Efficient Video Generative Modeling
      Python
      •
      MIT License
      •300•0•0•0•Updated Oct 21, 2024Oct 21, 2024

    • llama-stack

      Public
      Model components of the Llama Stack APIs
      Python
      •
      MIT License
      •1.3k•0•0•0•Updated Oct 10, 2024Oct 10, 2024

    • ngx-http-auth-jwt-module

      Public
      Secure your NGINX locations with JWT
      Shell
      •
      MIT License
      •134•0•0•0•Updated Jun 17, 2024Jun 17, 2024

    • deepctl

      Public
      Command line tool for Deep Infra cloud ML inference service
      Rust
      •
      Apache License 2.0
      •3•34•2•0•Updated Jun 10, 2024Jun 10, 2024

    • langchainjs

      Public
      🦜🔗 Build context-aware reasoning applications 🦜🔗
      TypeScript
      •
      MIT License
      •3.1k•0•0•0•Updated May 31, 2024May 31, 2024

    • deepinfra-node

      Public
      Official TypeScript wrapper for DeepInfra Inference API
      javascriptapiwrapper
      javascriptapiwrappertypescriptdeep-learningapi-clientllmllm-inference
      TypeScript
      •
      MIT License
      •3•20•6•2•Updated May 13, 2024May 13, 2024
    ProTip! When viewing an organization's repositories, you can use the props. filter to filter by custom property.