Protocol Reference
==================

This document describes the MT5 WebSocket binary protocol as reverse-engineered
from the official MetaTrader 5 Web Terminal JavaScript client. All byte orders
are **little-endian** unless stated otherwise.

Overview
--------

The MT5 Web Terminal communicates with the trade server over a single WebSocket
connection (default ``wss://web.metatrader.app/terminal``). Every WebSocket
message is a **binary frame** consisting of:

1. An 8-byte outer header.
2. An AES-CBC encrypted body containing a command frame.

The protocol is strictly request/response with the addition of server-initiated
**push notifications** for real-time data (ticks, order book, trade updates,
account changes).

Frame Format
------------

Outer Frame
~~~~~~~~~~~

Every WebSocket message has a fixed 8-byte header followed by the encrypted
body::

    +-----------+-----------+---------------------------+
    | Offset    | Size      | Description               |
    +-----------+-----------+---------------------------+
    | 0         | 4 bytes   | body_length (uint32 LE)   |
    | 4         | 4 bytes   | version (uint32 LE = 1)   |
    | 8         | N bytes   | encrypted body            |
    +-----------+-----------+---------------------------+

- ``body_length`` is the number of bytes that follow the header (i.e.
  ``len(encrypted_body)``).
- ``version`` is always ``1`` (``OUTER_PROTOCOL_VERSION``).

The outer frame is constructed by :func:`pymt5.protocol.pack_outer` and parsed
by :func:`pymt5.protocol.unpack_outer`.

Inner Command Frame
~~~~~~~~~~~~~~~~~~~

After decryption, the inner body has the following layout::

    +-----------+-----------+---------------------------+
    | Offset    | Size      | Description               |
    +-----------+-----------+---------------------------+
    | 0         | 2 bytes   | random prefix (uint16)    |
    | 2         | 2 bytes   | command_id (uint16 LE)    |
    | 4         | N bytes   | command payload            |
    +-----------+-----------+---------------------------+

**Request frames** (client to server):

- Bytes 0-1 are random (generated by ``os.urandom(2)``), providing minimal
  replay resistance.
- Bytes 2-3 are the ``command_id`` identifying the operation.
- Bytes 4+ are the serialized command payload (may be empty).

**Response frames** (server to client):

- Bytes 0-1 are echoed from the request (ignored by the parser).
- Bytes 2-3 are the ``command_id``.
- Byte 4 is the ``response_code`` (0 = success).
- Bytes 5+ are the response body.

Key Exchange
------------

The connection begins with an **unencrypted bootstrap** phase. The initial
cipher uses a hard-coded obfuscated key (``INITIAL_KEY_OBFUSCATED`` in
``constants.py``) decoded via a simple character-shift algorithm.

Bootstrap Flow (cmd=0)
~~~~~~~~~~~~~~~~~~~~~~

1. Client connects to the WebSocket endpoint.
2. Client sends ``CMD_BOOTSTRAP`` (cmd=0) with a 64-byte zero token as payload,
   encrypted with the initial (hard-coded) AES key.
3. Server responds with:

   - ``code=0`` on success.
   - Response body layout::

       +--------+--------+---------------------------------+
       | Offset | Size   | Description                     |
       +--------+--------+---------------------------------+
       | 0      | 2      | reserved                        |
       | 2      | 64     | session token                   |
       | 66     | 16/24  | AES session key (128 or 192bit) |
       +--------+--------+---------------------------------+

4. Client replaces the initial cipher with a new ``AESCipher`` initialized from
   the session key. All subsequent messages use this session key.

AES-CBC Encryption
~~~~~~~~~~~~~~~~~~

All frames (after the outer header) are encrypted using AES-CBC:

- **Mode**: CBC (Cipher Block Chaining).
- **IV**: Always 16 zero bytes (``b"\x00" * 16``).
- **Padding**: PKCS7 with a 128-bit block size.
- **Key size**: 128-bit (16 bytes) or 192-bit (24 bytes), as provided by the
  server during bootstrap.

The :class:`pymt5.crypto.AESCipher` class wraps the ``cryptography`` library
primitives.

Session Lifecycle
-----------------

The following text diagram shows the typical connection state transitions::

    DISCONNECTED
        |
        v
    CONNECTING ---- WebSocket open ---->  (send bootstrap cmd=0)
        |
        | (receive bootstrap response, extract session key)
        v
      READY ---- (optional) cmd=29 init_session ---->
        |
        | cmd=28 login
        v
    AUTHENTICATED ---- trading, data subscriptions ---->
        |
        | cmd=2 logout (or disconnect)
        v
    DISCONNECTED

Transport State Machine
~~~~~~~~~~~~~~~~~~~~~~~

The :class:`pymt5.transport.TransportState` enum tracks these states:

- ``DISCONNECTED`` -- No active connection.
- ``CONNECTING`` -- WebSocket open in progress and bootstrap handshake.
- ``READY`` -- Bootstrap complete, session key exchanged, commands accepted.
- ``CLOSING`` -- Graceful shutdown initiated.
- ``ERROR`` -- Unexpected disconnection or protocol error.

On unexpected disconnect, if ``auto_reconnect`` is enabled, the client enters a
reconnect loop with exponential backoff and jitter:
``min(base_delay * 2^(attempt-1) + random(0, base_delay), max_delay)``.

Command ID Catalog
------------------

The following table lists all command IDs defined in ``pymt5/constants.py``.
Commands marked "push" are server-initiated notifications.

.. list-table:: Command IDs
   :header-rows: 1
   :widths: 10 30 60

   * - ID
     - Constant
     - Description
   * - 0
     - ``CMD_BOOTSTRAP``
     - Bootstrap handshake: exchange session token and AES key.
   * - 2
     - ``CMD_LOGOUT``
     - End the authenticated session.
   * - 3
     - ``CMD_GET_ACCOUNT``
     - Retrieve account info (balance, equity, margin, leverage).
   * - 4
     - ``CMD_GET_POSITIONS_ORDERS``
     - Retrieve open positions and pending orders.
   * - 5
     - ``CMD_GET_TRADE_HISTORY``
     - Retrieve historical deals.
   * - 6
     - ``CMD_GET_SYMBOLS``
     - Retrieve basic symbol list (name, id, digits, path).
   * - 7
     - ``CMD_SUBSCRIBE_TICKS``
     - Subscribe to real-time tick data for given symbol IDs.
   * - 8
     - ``CMD_TICK_PUSH``
     - **Push**: Real-time tick data for subscribed symbols.
   * - 9
     - ``CMD_GET_SYMBOL_GROUPS``
     - Retrieve symbol group names.
   * - 10
     - ``CMD_TRADE_UPDATE_PUSH``
     - **Push**: Trade state change (balance update or order transaction).
   * - 11
     - ``CMD_GET_RATES``
     - Retrieve OHLCV bars (klines) for a symbol and timeframe.
   * - 12
     - ``CMD_TRADE_REQUEST``
     - Submit a trade request (market, pending, modify, close).
   * - 13
     - ``CMD_SYMBOL_UPDATE_PUSH``
     - **Push**: Symbol property change (spread, session, etc.).
   * - 14
     - ``CMD_ACCOUNT_UPDATE_PUSH``
     - **Push**: Account balance/margin/equity change.
   * - 15
     - ``CMD_LOGIN_STATUS_PUSH``
     - **Push**: Login status change (forced logout, session expiry).
   * - 17
     - ``CMD_SYMBOL_DETAILS_PUSH``
     - **Push**: Extended symbol quote data (greeks, session stats).
   * - 18
     - ``CMD_GET_FULL_SYMBOLS``
     - Retrieve full symbol specifications (contract details).
   * - 19
     - ``CMD_TRADE_RESULT_PUSH``
     - **Push**: Asynchronous trade execution result.
   * - 20
     - ``CMD_GET_SPREADS``
     - Retrieve spread data for symbols.
   * - 22
     - ``CMD_SUBSCRIBE_BOOK``
     - Subscribe to order book (depth-of-market) for given symbol IDs.
   * - 23
     - ``CMD_BOOK_PUSH``
     - **Push**: Order book update for subscribed symbols.
   * - 24
     - ``CMD_CHANGE_PASSWORD``
     - Change the account password.
   * - 27
     - ``CMD_VERIFY_CODE``
     - Verify an email/phone confirmation code.
   * - 28
     - ``CMD_LOGIN``
     - Authenticate with login, password, and optional OTP.
   * - 29
     - ``CMD_INIT``
     - Initialize session (pre-login handshake for cmd=28).
   * - 30
     - ``CMD_OPEN_DEMO``
     - Open a demo account.
   * - 34
     - ``CMD_GET_SYMBOLS_GZIP``
     - Retrieve symbols (gzip-compressed variant).
   * - 39
     - ``CMD_OPEN_REAL``
     - Open a real (live) trading account.
   * - 40
     - ``CMD_SEND_VERIFY_CODES``
     - Request email/phone verification codes to be sent.
   * - 41
     - ``CMD_TRADER_PARAMS``
     - Retrieve trader parameter strings.
   * - 42
     - ``CMD_NOTIFY``
     - Retrieve server notifications.
   * - 43
     - ``CMD_OTP_SETUP``
     - Set up or manage OTP (one-time password / 2FA).
   * - 44
     - ``CMD_GET_CORPORATE_LINKS``
     - Retrieve corporate links (help, support URLs).
   * - 51
     - ``CMD_PING``
     - Heartbeat / keep-alive ping.

Field Type Encoding
-------------------

The :class:`pymt5.protocol.SeriesCodec` serializes and deserializes fields
according to the ``PROP_*`` type constants. All multi-byte integers and floats
use **little-endian** byte order.

.. list-table:: Field Types
   :header-rows: 1
   :widths: 15 10 10 10 55

   * - Constant
     - ID
     - Size
     - Endian
     - Description
   * - ``PROP_I8``
     - 1
     - 1 byte
     - N/A
     - Signed 8-bit integer (``struct "<b"``).
   * - ``PROP_I16``
     - 2
     - 2 bytes
     - LE
     - Signed 16-bit integer (``struct "<h"``).
   * - ``PROP_I32``
     - 3
     - 4 bytes
     - LE
     - Signed 32-bit integer (``struct "<i"``).
   * - ``PROP_U8``
     - 4
     - 1 byte
     - N/A
     - Unsigned 8-bit integer (``struct "<B"``).
   * - ``PROP_U16``
     - 5
     - 2 bytes
     - LE
     - Unsigned 16-bit integer (``struct "<H"``).
   * - ``PROP_U32``
     - 6
     - 4 bytes
     - LE
     - Unsigned 32-bit integer (``struct "<I"``).
   * - ``PROP_F32``
     - 7
     - 4 bytes
     - LE
     - IEEE 754 single-precision float (``struct "<f"``).
   * - ``PROP_F64``
     - 8
     - 8 bytes
     - LE
     - IEEE 754 double-precision float (``struct "<d"``).
   * - ``PROP_TIME``
     - 9
     - 8 bytes
     - LE
     - Windows FILETIME (100ns ticks since 1601-01-01). Converted to/from Unix
       milliseconds by the codec.
   * - ``PROP_STRING``
     - 10
     - N bytes
     - N/A
     - Fixed-length ASCII string (``propLength`` required). Padded with null
       bytes, decoded as Latin-1.
   * - ``PROP_FIXED_STRING``
     - 11
     - N bytes
     - LE
     - Fixed-length UTF-16LE string (``propLength`` required). Padded with null
       bytes, terminated at first null character pair.
   * - ``PROP_BYTES``
     - 12
     - N bytes
     - N/A
     - Raw byte buffer (``propLength`` required for fixed-size fields). Padded
       with null bytes when ``propLength`` is specified.
   * - ``PROP_I64``
     - 17
     - 8 bytes
     - LE
     - Signed 64-bit integer (``int.from_bytes(..., signed=True)``).
   * - ``PROP_U64``
     - 18
     - 8 bytes
     - LE
     - Unsigned 64-bit integer (``int.from_bytes(..., signed=False)``).

Tick Push Notifications (cmd=8)
-------------------------------

When symbols are subscribed via ``CMD_SUBSCRIBE_TICKS`` (cmd=7), the server
streams tick updates as ``CMD_TICK_PUSH`` (cmd=8) push notifications.

Tick Body Format
~~~~~~~~~~~~~~~~

The response body contains a 4-byte count followed by repeated tick records::

    +--------+--------+---------------------------+
    | Offset | Size   | Description               |
    +--------+--------+---------------------------+
    | 0      | 4      | tick_count (uint32 LE)    |
    | 4      | 42 * N | tick records              |
    +--------+--------+---------------------------+

Each tick record (42 bytes) follows the ``TICK_SCHEMA``:

.. list-table:: Tick Record Fields
   :header-rows: 1
   :widths: 5 15 15 65

   * - #
     - Field
     - Type
     - Description
   * - 0
     - symbol_id
     - U32
     - Symbol identifier.
   * - 1
     - tick_time
     - I32
     - Tick time in Unix seconds.
   * - 2
     - fields
     - U32
     - Bitmask indicating which fields are populated.
   * - 3
     - bid
     - F64
     - Best bid price.
   * - 4
     - ask
     - F64
     - Best ask price.
   * - 5
     - last
     - F64
     - Last trade price.
   * - 6
     - tick_volume
     - I64
     - Tick volume.
   * - 7
     - time_ms_delta
     - U32
     - Millisecond offset from ``tick_time``.
   * - 8
     - flags
     - U16
     - Tick flags (bid changed, ask changed, etc.).

Book Push Notifications (cmd=23)
--------------------------------

When symbols are subscribed via ``CMD_SUBSCRIBE_BOOK`` (cmd=22), the server
streams order book updates as ``CMD_BOOK_PUSH`` (cmd=23) push notifications.

Book Body Format
~~~~~~~~~~~~~~~~

The response body contains one or more symbol book entries, each consisting of a
header followed by bid and ask level arrays::

    +--------+--------+------------------------------------------+
    | Offset | Size   | Description                              |
    +--------+--------+------------------------------------------+
    | 0      | 22     | Book header (BOOK_HEADER_SCHEMA)         |
    | 22     | 16 * B | Bid levels (B = bid_count from header)   |
    | ...    | 16 * A | Ask levels (A = ask_count from header)   |
    +--------+--------+------------------------------------------+

**Book Header** (22 bytes):

.. list-table:: Book Header Fields
   :header-rows: 1
   :widths: 5 15 10 70

   * - #
     - Field
     - Type
     - Description
   * - 0
     - symbol_id
     - U32
     - Symbol identifier.
   * - 1
     - field1
     - I32
     - Reserved field.
   * - 2
     - field2
     - I32
     - Reserved field.
   * - 3
     - bid_count
     - U32
     - Number of bid levels following.
   * - 4
     - ask_count
     - U32
     - Number of ask levels following.
   * - 5
     - flags
     - U16
     - Book update flags.

**Book Level** (16 bytes each):

.. list-table:: Book Level Fields
   :header-rows: 1
   :widths: 5 15 10 70

   * - #
     - Field
     - Type
     - Description
   * - 0
     - price
     - F64
     - Price level.
   * - 1
     - volume
     - I64
     - Volume at this price level.

Error Codes
-----------

Trade requests (cmd=12) return a ``retcode`` in the response indicating success
or failure. The following table lists all ``TRADE_RETCODE_*`` constants:

.. list-table:: Trade Return Codes
   :header-rows: 1
   :widths: 10 25 65

   * - Code
     - Constant
     - Description
   * - 10004
     - ``TRADE_RETCODE_REQUOTE``
     - Requote.
   * - 10006
     - ``TRADE_RETCODE_REJECT``
     - Request rejected.
   * - 10007
     - ``TRADE_RETCODE_CANCEL``
     - Request cancelled by trader.
   * - 10008
     - ``TRADE_RETCODE_PLACED``
     - Order placed.
   * - 10009
     - ``TRADE_RETCODE_DONE``
     - Request completed (done).
   * - 10010
     - ``TRADE_RETCODE_DONE_PARTIAL``
     - Request partially completed.
   * - 10011
     - ``TRADE_RETCODE_ERROR``
     - Request processing error.
   * - 10012
     - ``TRADE_RETCODE_TIMEOUT``
     - Request cancelled by timeout.
   * - 10013
     - ``TRADE_RETCODE_INVALID``
     - Invalid request.
   * - 10014
     - ``TRADE_RETCODE_INVALID_VOLUME``
     - Invalid volume.
   * - 10015
     - ``TRADE_RETCODE_INVALID_PRICE``
     - Invalid price.
   * - 10016
     - ``TRADE_RETCODE_INVALID_STOPS``
     - Invalid stops.
   * - 10017
     - ``TRADE_RETCODE_TRADE_DISABLED``
     - Trade disabled.
   * - 10018
     - ``TRADE_RETCODE_MARKET_CLOSED``
     - Market closed.
   * - 10019
     - ``TRADE_RETCODE_NO_MONEY``
     - Not enough money.
   * - 10020
     - ``TRADE_RETCODE_PRICE_CHANGED``
     - Price changed.
   * - 10021
     - ``TRADE_RETCODE_PRICE_OFF``
     - No quotes for request processing.
   * - 10022
     - ``TRADE_RETCODE_INVALID_EXPIRATION``
     - Invalid order expiration.
   * - 10023
     - ``TRADE_RETCODE_ORDER_CHANGED``
     - Order state changed.
   * - 10024
     - ``TRADE_RETCODE_TOO_MANY_REQUESTS``
     - Too many requests.
   * - 10025
     - ``TRADE_RETCODE_NO_CHANGES``
     - No changes in request.
   * - 10026
     - ``TRADE_RETCODE_SERVER_DISABLES_AT``
     - Auto-trading disabled by server.
   * - 10027
     - ``TRADE_RETCODE_CLIENT_DISABLES_AT``
     - Auto-trading disabled by client.
   * - 10028
     - ``TRADE_RETCODE_LOCKED``
     - Request locked for processing.
   * - 10029
     - ``TRADE_RETCODE_FROZEN``
     - Order/position frozen.
   * - 10030
     - ``TRADE_RETCODE_INVALID_FILL``
     - Invalid order filling type.
   * - 10031
     - ``TRADE_RETCODE_CONNECTION``
     - No connection with trade server.
   * - 10032
     - ``TRADE_RETCODE_ONLY_REAL``
     - Operation allowed only for live accounts.
   * - 10033
     - ``TRADE_RETCODE_LIMIT_ORDERS``
     - Pending orders limit reached.
   * - 10034
     - ``TRADE_RETCODE_LIMIT_VOLUME``
     - Volume limit for symbol reached.
   * - 10035
     - ``TRADE_RETCODE_INVALID_ORDER``
     - Invalid or prohibited order type.
   * - 10036
     - ``TRADE_RETCODE_POSITION_CLOSED``
     - Position already closed.

Volume Encoding
---------------

MT5 uses integer volume encoding: the wire value is ``volume * 10000``. For
example, 0.01 lots is transmitted as ``100``, and 1.0 lots as ``10000``. The
pymt5 client handles this conversion transparently in trading helper methods.

Rate Bar Format (cmd=11)
------------------------

Historical OHLCV bars returned by ``CMD_GET_RATES`` (cmd=11) have a 48-byte
per-bar layout:

.. list-table:: Rate Bar Fields
   :header-rows: 1
   :widths: 5 15 10 70

   * - #
     - Field
     - Type
     - Description
   * - 0
     - time
     - I32
     - Bar open time (Unix seconds).
   * - 1
     - open
     - F64
     - Open price.
   * - 2
     - high
     - F64
     - High price.
   * - 3
     - low
     - F64
     - Low price.
   * - 4
     - close
     - F64
     - Close price.
   * - 5
     - tick_volume
     - I64
     - Tick volume during the bar period.
   * - 6
     - spread
     - I32
     - Spread at bar close.