genai.extensions.localserver.local_api_server module

class genai.extensions.localserver.local_api_server.ApiAuthMiddleware

Bases: BaseHTTPMiddleware

__init__(app, api_key=None, insecure=False)

Parameters:

• app (FastAPI) –

• api_key (str | None) –

• insecure (bool) –

async dispatch(request, call_next)

Parameters:

request (Request) –

class genai.extensions.localserver.local_api_server.ApiErrorMiddleware

Bases: BaseHTTPMiddleware

__init__(app, dispatch=None)

Parameters:

• app (FastAPI) –

• dispatch (Callable[[Request, Callable[[Request], Awaitable[Response]]], Awaitable[Response]] | None) –

Return type:

None

async dispatch(request, call_next)

Parameters:

• request (Request) –

• call_next (Callable[[Request], Awaitable[Response]]) –

Return type:

Response

class genai.extensions.localserver.local_api_server.LocalLLMServer

Bases: object

__init__(models, port=8080, interface='0.0.0.0', api_key=None, insecure_api=False)

Parameters:

• models (list[type[LocalModel]]) –

• port (int) –

• interface (str) –

• api_key (str | None) –

• insecure_api (bool) –

get_credentials()

run_locally()

start_server()

stop_server()