|
Kong Open Sources New AI Gateway to Help Developers Easily Build Multi-LLM Apps
Friday, February 23, 2024
Company Unveils Open Source 'No-Code' Plugin Suite Featuring Support for Multi-LLMs, Advanced Prompt Engineering, and AI Analytics for Fast and Secure High-Performance AI Adoption
SAN FRANCISCO, Feb. 15, 2024 /PRNewswire/ -- Kong Inc., a leading developer of cloud API technologies, today announced a suite of open-source AI plugins for Kong Gateway 3.6 that can turn any Kong Gateway deployment into an AI Gateway, offering unprecedented support for multi-Language Learning Models (LLMs) integration. By upgrading to Kong Gateway 3.6, available today, users can access a suite of six new plugins that are entirely focused on AI and LLM usage. This will enable developers who want to integrate one or more LLMs into their products to be more productive and ship AI capabilities faster, while at the same time offering architects and platform teams a secure solution that ensures visibility, control and compliance on every AI request sent by the teams. Due to the tight integration with Kong Gateway, it will now be possible to easily orchestrate AI flows in the cloud or on self-hosted LLMs with industry leading performance and low latency, which are critical to the performance of AI-based applications.
"Today marks a significant milestone in our journey towards democratizing AI for developers and enterprises worldwide. By open-sourcing this suite of innovative AI capabilities, including no-code AI plugins, we're removing the barriers to AI adoption and making it possible for developers to leverage multiple LLMs effortlessly and ship AI powered applications faster. At the same time, we're providing governance and visibility to all the AI traffic that is being generated by an organization," said Marco Palladino, Chief Technology Officer and Co-Founder, Kong Inc.
Multi-LLM integrations, AI analytics, advanced engineer prompts, and more
Builders want the best AI for their use case, need to lower the time it takes to integrate different LLMs, and want to rapidly iterate and build new capabilities without having to manage the specific cross-cutting requirements around AI usage.
By upgrading to Kong Gateway 3.6, AI builders can access this new suite of plugins entirely focused on AI and LLM usage. The suite of open source plugins delivers a range of new capabilities, including:
-- Multi-LLM Integration: Kong Inc.'s "ai-proxy" plugin enables seamless
integration of multiple Large Language Model (LLM) implementations,
offering native support for industry leaders including OpenAI, Azure AI,
Cohere, Anthropic, Mistral, and LLAMA. The standardized interface allows
for simple switching between LLMs without modifying application code,
facilitating the use of diverse models and rapid prototyping.
-- Central AI Credential Management: The "ai-proxy" helps ensure secure and
centralized storage of AI credentials within Kong Gateway. This design
negates the need for credentials within applications, streamlining
credential rotation and updates directly from the gateway.
-- Layer 7 AI Metrics Collection: Leveraging the "ai-proxy" plugin, users
can now capture detailed Layer 7 AI analytics. This includes metrics
such as request and response token counts, along with usage data for LLM
providers and models. Integration with third-party platforms like
Datadog, New Relic, and existing logging plugins in Kong Gateway, like
TCP, Syslog, Prometheus, is supported, enriching observability and
offering insights into developer preferences.
-- No-Code AI Integrations: With the "ai-request-transformer" and
"ai-response-transformer" plugins, AI capabilities are injected into API
requests and responses without a single line of code. This allows for
on-the-fly transformations like real-time API response translations for
internationalization, enriching and converting API traffic effortlessly.
-- Advanced AI Prompt Engineering: Kong is introducing three new plugins
dedicated to sophisticated prompt engineering. The "ai-prompt-template"
facilitates the creation and central management of prompt templates
within Kong Gateway, enhancing the agility of prompt updates without
application code changes and helping ensure compliance through approved
template processes.
-- AI Prompt Decoration: The "ai-prompt-decorator" plugin allows for the
consistent configuration of AI prompt contexts, automating the inclusion
of rules and instructions with each AI request to enforce organizational
compliance and restrict discussions on sensitive topics.
-- AI Prompt Firewall: The "ai-prompt-guard" offers a governance layer,
establishing rules to authorize or block free-form prompts created by
applications. This helps ensure that prompts adhere to approved
standards before being transmitted to LLM providers.
-- Comprehensive AI Egress with Extensive Features: The integration of
these AI capabilities within Kong Gateway centralizes the management,
security, and monitoring of AI traffic. It leverages over 1,000+
existing official and community plugins for robust access control, rate
limiting, and the creation of advanced traffic control rules. The AI
Gateway is equipped from day one with all Kong Gateway features, making
it, we believe, the most capable in the AI ecosystem.
With this release, Kong's expertise in modern API infrastructure now extends to AI-driven use cases. After extensive research and collaboration with select customers and users of Kong Gateway, these newly released plugins address what we believe are the most prevalent AI use cases. We anticipate releasing additional AI features in the future and always welcome user feedback.
For more information:
-- AI Gateway Webinar: Join our upcoming Fireside Chat webinar with Marco
Palladino, Kong's co-founder and CTO, on March 13, 2024, at 8:00 AM PDT
to explore how Kong is paving the way for seamless AI adoption. Register
here.
-- Getting Started with Kong's AI Gateway: Here, you'll find a
comprehensive quick start guide, insightful demo videos, and direct
download links to equip you with the tools for swift and secure AI
adoption. Leverage the power of Kong's AI Gateway to enhance your
applications and services. Get started today.
About Kong
Kong Inc., a leading developer of cloud API technologies, is on a mission to enable companies around the world to become "API-first." Kong helps organizations globally -- from startups to Fortune 500 enterprises -- unleash developer productivity, build securely and accelerate time to market. For more information about Kong, please visit www.konghq.com or follow us on X @thekonginc.
View original content to download multimedia:https://www.prnewswire.com/news-releases/kong-open-sources-new-ai-gateway-to-help-developers-easily-build-multi-llm-apps-302062656.html
SOURCE Kong Inc.
|
|
|
|
|
|
 |
Dream Vacations Transforms Travel Booking Experience with New Website | Mar 28, 2025
|
|
Land id(TM) Partners with The Land Report to Profile America's Largest Landowners | Mar 28, 2025
|
|
New medical technology pilot environment established in Finland aims to slash market entry time of patient-friendly solutions | Mar 28, 2025
|
|
Playcasino.co.nz Releases Comprehensive Guide to Online Casino Payment Methods for NZ Players | Mar 28, 2025
|
|
MetAI to Debut AI-Powered Controller Simulator at NVIDIA GTC 2025, Advancing Digital Twins for Industrial Automation | Mar 28, 2025
|
|
Netflix Worldwide Exclusive Streaming Anime"The Summer Hikaru Died" Reveals Main Trailer and New Cast Members, Yumiri Hanamori, Wakana Kowaka, and Chikahiro Kobayashi | Mar 28, 2025
|
|
Constructor Unveils Retail Media Suite with Personalized Sponsored Listings and Retail Media Network Integrations | Mar 28, 2025
|
|
Appcast Named a 2025 Google Premier Partner | Mar 28, 2025
|
|
Hy-Vee RedMedia Partners with Instacart to Further Retail Media Capabilities | Mar 28, 2025
|
|
New Website Developed by Flightpath, A Ruder Finn Company Launched by Goya Foods | Mar 28, 2025
|
|
|
|
