AnythingLLM

this only applies to local LLM providers. It has no impact on cloud LLMs like OpenAI, Anthropic, or Azure.

We have completely overhauled how @agent tool calling works. Now, we will leverage the new native tool calling abilities of your LLM provider and model.

What this means for you:

• You can now run complex, multi-step tool calls with your LLM provider and model.
• Your model will now continue to work until your final response is generated or determined to be complete.
• You will get 100x better responses from even small tool-calling models

We have implemented safeguards as well to prevent infinite loops with a maximum of 10 tool calls per response to prevent runaway tasks.

Limitations

Most providers do not allow us to probe for if a model supports native tool calling.

The following local LLM providers will automatically support native tool calling if your model supports it:

• Default Built in LLM Provider (AnythingLLM Default)
• Ollama
• LM Studio

For others, you will need to set an ENV variable to enable native tool calling for supported providers.

• Generic OpenAI
• Groq
• AWS Bedrock
• Lemonade
• LiteLLM
• Local AI
• OpenRouter

This can be set via the PROVIDER_SUPPORTS_NATIVE_TOOL_CALLING environment variable.

PROVIDER_SUPPORTS_NATIVE_TOOL_CALLING="bedrock,generic-openai,groq,lemonade,litellm,local-ai,openrouter"

Lemonade by AMD Integration

Lemonade by AMD is an open-source local model runtime that optimizes performance and efficiency for local models (LLM, ASR, TTS, Image Generation, etc.) for all types of hardware including AMD GPUs and NPUs.

We have added first class support so you can use your local models running via Lemonade within AnythingLLM for the best application experience on top of your local hardware.

What’s Changed

• fix: typo in contribution guidelines, update project metadata and pull_request_temp…md by @dipanshurdev in https://github.com/Mintplex-Labs/anything-llm/pull/5010
• feat: update light mode UI sidebar by @angelplusultra in https://github.com/Mintplex-Labs/anything-llm/pull/4996
• fix(frontend): fix event listener memory leak in useIsDisabled hook by @dipanshurdev in https://github.com/Mintplex-Labs/anything-llm/pull/5027
• feat: dedicated dark theme option with system preference support by @angelplusultra in https://github.com/Mintplex-Labs/anything-llm/pull/5007
• Implement new home page redesign by @shatfield4 in https://github.com/Mintplex-Labs/anything-llm/pull/4931
• fix: GitLab connector infinite loop and rate limit crash for large repos by @angelplusultra in https://github.com/Mintplex-Labs/anything-llm/pull/5021
• fix: add password character validation to onboarding single-user setup by @angelplusultra in https://github.com/Mintplex-Labs/anything-llm/pull/5037
• Native Tool calling by @timothycarambat in https://github.com/Mintplex-Labs/anything-llm/pull/5071
• fix: resolve Gemini agent 400 error on tool call responses by @angelplusultra in https://github.com/Mintplex-Labs/anything-llm/pull/5054
• fix: prevent CMD/CTRL+Arrow scroll from overriding textarea cursor movement by @angelplusultra in https://github.com/Mintplex-Labs/anything-llm/pull/5053
• Normalize scraper runtimeargs for bulk-scraper by @timothycarambat in https://github.com/Mintplex-Labs/anything-llm/pull/5083
• Lemonade integration by @timothycarambat in https://github.com/Mintplex-Labs/anything-llm/pull/5077

New Contributors

• @dipanshurdev made their first contribution in https://github.com/Mintplex-Labs/anything-llm/pull/5010

Full Changelog: https://github.com/Mintplex-Labs/anything-llm/compare/v1.11.0…v1.11.1