Integration
Microservices Architecture
Microservices architecture structures an application as a collection of small, independently deployable services that each own a single business capability and communicate over lightweight APIs. It contrasts with monolithic architectures, where all functions share a single deployable unit.
AI systems built on microservices isolate model inference, retrieval, guardrails, and business logic into separate services that can be scaled, updated, and monitored independently. The tradeoff is operational complexity: microservices require service discovery, distributed tracing, and resilient inter-service communication. For most enterprise AI deployments, a modular monolith is preferable to microservices until scale demands otherwise.
Related terms
- API Gateway (AI) — An AI API gateway is a proxy layer that sits between internal applications and external AI model APIs, enforcing rate limits, cost controls, authentication, PII redaction, audit logging, and fallback routing across multiple model providers.
- LLM Integration — LLM integration is the work of embedding a large language model into existing business systems — CRM, ERP, ticketing, document repositories — so that AI capabilities are accessible from the tools employees already use, with proper authentication, audit logging, and data isolation.
- Event Streaming — Event streaming is the continuous capture, transmission, and processing of event records in real time through a distributed log — Apache Kafka being the dominant platform.