genai.extensions.localserver.local_api_server module

class genai.extensions.localserver.local_api_server.ApiAuthMiddleware

Bases: BaseHTTPMiddleware

__init__(app, api_key=None, insecure=False)

Parameters:

• app (FastAPI)

• api_key (str | None)

• insecure (bool)

async dispatch(request, call_next)

Parameters:

request (Request)

class genai.extensions.localserver.local_api_server.ApiErrorMiddleware

Bases: BaseHTTPMiddleware

__init__(app, dispatch=None)

Parameters:

• app (FastAPI)

• dispatch (Callable[[Request, Callable[[Request], Awaitable[Response]]], Awaitable[Response]] | None)

Return type:

None

async dispatch(request, call_next)

Parameters:

• request (Request)

• call_next (Callable[[Request], Awaitable[Response]])

Return type:

Response

class genai.extensions.localserver.local_api_server.LocalLLMServer

Bases: object

__init__(models, port=8080, interface='0.0.0.0', api_key=None, insecure_api=False)

Parameters:

• models (list[type[LocalModel]])

• port (int)

• interface (str)

• api_key (str | None)

• insecure_api (bool)

get_credentials()

run_locally()

start_server()

stop_server()