v0.4.1

What's Changed

• Add pre-commit CI config by @amrit110 in #20
• Add dependabot.yml file by @amrit110 in #23
• Update jinja2 and fix poetry.lock file by @amrit110 in #25
• Bump up vllm to 0.7.0 by @amrit110 in #27
• Add CI check for PRs submitted to develop branch by @amrit110 in #26
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #22
• Migrate to uv, add lint fixes, improvements and some unit tests by @amrit110 in #28
• onboard BGE-Base-EN-v1.5 and All-MiniLM-L6-v2 by @kohankhaki in #29
• Add docs for vector-inference by @amrit110 in #30
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #31
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #32
• Update docs_build.yml by @amrit110 in #33
• Update dependencies by @fcogidi in #34

New Contributors

• @pre-commit-ci made their first contribution in #22

Full Changelog: v0.4.0...v0.4.1

v0.4.0.post1

• Fix wrong dependency
• Updated README files

v0.4.0

• Onboarded various new models and new model types: text embedding model and reward reasoning model.
• Added metrics command that streams performance metrics for inference server.
• Enabled more launch command options: --max-num-seqs, --model-weights-parent-dir, --pipeline-parallelism, --enforce-eager.
• Improved support for launching custom models.
• Improved command response time.
• Improved visuals for list command.

v0.3.3

• Added missing package in decencies
• Fixed pre-commit hooks
• Linted and formatted code
• Updated outdated examples

v0.3.2

• Add support for custom models, users can now launch custom models as long as the model architecture is supported by vllm
• Minor update multi-node job launching to better support custom models
• Add Llama3-OpenBioLLM-70B to supported model list

v0.3.1

• Add model-name argument to list command to show default setup of a specific supported model
• Improved command option descriptions
• Restructured models directory
• Add some default values for using a custom model

v0.3.0

• Added vec-inf CLI:

  • Install vec-inf via pip
  • launch command to launch models
  • status command to check inference server status
  • shutdown command to stop inference server
  • list command to see all available models

• Upgraded vllm to 0.5.4

• Added support for new model families:

  • Llama 3.1 (Including 405B)
  • Gemma 2
  • Phi 3
  • Mistral Large

v0.2.1

• Add CodeLlama
• Update model variant names for Llama 2 in README

v0.2.0

• Update default environment to use singularity container, added associated Dockerfile
• Update vLLM to 0.5.0 and added VLM support (LLaVa-1.5 and LLaVa-NEXT) and updated example scripts
• Refactored repo structure for simpler model onboard and update process

v0.1.1

• Update vllm to 0.4.2, which resolves the flash attention package not found issue
• Update instructions for using the default environment to prevent/resolve NCCL not found error