pymllm.service.network
======================

.. py:module:: pymllm.service.network


Attributes
----------

.. autoapisummary::

   pymllm.service.network.MODEL_SESSION_CREATED
   pymllm.service.network.app


Classes
-------

.. autoapisummary::

   pymllm.service.network.ChatCompletionRequest


Functions
---------

.. autoapisummary::

   pymllm.service.network.lifespan
   pymllm.service.network.predict


Module Contents
---------------

.. py:data:: MODEL_SESSION_CREATED

.. py:function:: lifespan(app)
   :async:


   Starts the background service on application startup and stops it on shutdown.


.. py:data:: app

.. py:class:: ChatCompletionRequest(/, **data)

   Bases: :py:obj:`pydantic.BaseModel`


   !!! abstract "Usage Documentation"
       [Models](../concepts/models.md)

   A base class for creating Pydantic models.

   .. attribute:: __class_vars__

      The names of the class variables defined on the model.

   .. attribute:: __private_attributes__

      Metadata about the private attributes of the model.

   .. attribute:: __signature__

      The synthesized `__init__` [`Signature`][inspect.Signature] of the model.

   .. attribute:: __pydantic_complete__

      Whether model building is completed, or if there are still undefined fields.

   .. attribute:: __pydantic_core_schema__

      The core schema of the model.

   .. attribute:: __pydantic_custom_init__

      Whether the model has a custom `__init__` function.

   .. attribute:: __pydantic_decorators__

      Metadata containing the decorators defined on the model.
      This replaces `Model.__validators__` and `Model.__root_validators__` from Pydantic V1.

   .. attribute:: __pydantic_generic_metadata__

      Metadata for generic models; contains data used for a similar purpose to
      __args__, __origin__, __parameters__ in typing-module generics. May eventually be replaced by these.

   .. attribute:: __pydantic_parent_namespace__

      Parent namespace of the model, used for automatic rebuilding of models.

   .. attribute:: __pydantic_post_init__

      The name of the post-init method for the model, if defined.

   .. attribute:: __pydantic_root_model__

      Whether the model is a [`RootModel`][pydantic.root_model.RootModel].

   .. attribute:: __pydantic_serializer__

      The `pydantic-core` `SchemaSerializer` used to dump instances of the model.

   .. attribute:: __pydantic_validator__

      The `pydantic-core` `SchemaValidator` used to validate instances of the model.

   .. attribute:: __pydantic_fields__

      A dictionary of field names and their corresponding [`FieldInfo`][pydantic.fields.FieldInfo] objects.

   .. attribute:: __pydantic_computed_fields__

      A dictionary of computed field names and their corresponding [`ComputedFieldInfo`][pydantic.fields.ComputedFieldInfo] objects.

   .. attribute:: __pydantic_extra__

      A dictionary containing extra values, if [`extra`][pydantic.config.ConfigDict.extra]
      is set to `'allow'`.

   .. attribute:: __pydantic_fields_set__

      The names of fields explicitly set during instantiation.

   .. attribute:: __pydantic_private__

      Values of private attributes set on the model instance.


   .. py:attribute:: model
      :type:  str


   .. py:attribute:: messages
      :type:  list


   .. py:attribute:: stream
      :type:  bool
      :value: True



   .. py:attribute:: id
      :type:  str
      :value: None



   .. py:attribute:: enable_thinking
      :type:  bool
      :value: None



.. py:function:: predict(req)
   :async:


   OpenAI-compatible /v1/chat/completions endpoint.