Welcome to ventu’s documentation!

Serving the deep learning models easily.

Install

pip install ventu

Features

• Only need to implement Model(preprocess, postprocess, inference or batch_inference)
• request & response data validation using pydantic
• API document using SpecTree (when run with run_http)
• backend service using falcon supports both JSON and msgpack
• dynamic batching with batching using Unix domain socket or TCP
  • errors in one request won’t affect others in the same batch
• support all the runtime
• health check
• inference warm-up

Contents:

• Ventu API
  • Config
  • Protocol
  • HTTP service

Indices and tables

• Index
• Module Index
• Search Page