Iteration Plan v3

Project Status Summary 

Current version: v0.8.0 (post Phase 1-5 + cleanup rounds)

Codebase metrics:

Source code: ~4,900 lines across 23 modules
Test code: 830 tests, 99% coverage (2750/2789 statements)
CI: 9-matrix (3 Python versions x 3 OS)
Documentation: Sphinx + 8 examples + 2 iteration plans
Mypy: 0 errors (strict_optional, disallow_incomplete_defs enabled)
Ruff: 0 lint errors

Architecture (5 mixins + 1 core client):

MT5WebClient
+-- _MarketDataMixin      (845 lines)  -- symbols, ticks, bars, book, subscriptions
+-- _TradingMixin         (698 lines)  -- positions, orders, trade execution, validation
+-- _OrderHelpersMixin    (502 lines)  -- buy/sell/close/modify convenience methods
+-- _AccountMixin         (488 lines)  -- account info, OTP, verification
+-- _PushHandlersMixin    (334 lines)  -- push notification handlers

+ Infrastructure:
  +-- Transport            (228 lines)  -- WebSocket, state machine, rate limiter
  +-- Protocol codec       (270 lines)  -- dispatch-table based serialization
  +-- Schemas             (1076 lines)  -- 24 binary protocol schemas
  +-- Support modules      (340 lines)  -- events, logging, metrics, subscription, etc.

Completed in this round (v3 cleanup):

Fixed rate limiter race condition with asyncio.Lock (CRITICAL)
Fixed __version__ fallback from “0.7.0” to “0.8.0”
Exported MetricsCollector in __init__.py
Narrowed 4 broad except Exception catches to specific types
Added PYMT5_LOG_LEVEL environment variable support
Added iteration_plan_v2.rst to docs toctree
Removed stale aiohttp reference from docs
Updated tests to use custom exception types (830 tests, all passing)

Remaining Gaps 

Coverage gaps (39 uncovered lines):

transport.py:72,119,166,175-176,202,209    -- connection state transitions, error paths
exceptions.py:47-50                         -- TradeError attribute initialization
_market_data.py:200-202,398-399,406-407     -- exception paths in get_full_symbol_info()
_push_handlers.py:200,283-284              -- handler dispatch edge cases
protocol.py:246,251,256,260                 -- variable-length field error branches
_parsers.py:82,455-457,494                  -- timestamp coercion edge cases
client.py:272,301                           -- reconnection edge cases
_logging.py:32                              -- structlog import path
__init__.py:10-11                           -- ImportError fallback path

Code quality observations:

_market_data.py at 845 lines (slightly over 800 guideline)
schemas.py at 1076 lines (data-only, acceptable but monitor)
Symbol cache has no TTL/invalidation mechanism
No integration test framework (Phase 6.1 in v2)
No performance benchmarks (Phase 6.2 in v2)
No PyPI release automation (Phase 8.3 in v2)
No typed push handler callbacks (Phase 9.2 in v2)
No protocol version tracking (Phase 7.1 in v2)

Phase 10: Code Hardening (v0.9.0)

Goal: Close coverage gaps, improve error paths, prepare for v1.0.

10.1 Close coverage to 99.5%

Priority: HIGH

Target the 39 uncovered lines with focused tests:

transport.py (7 lines):

Line 72: connect() when ws is not None triggers close() first
Line 119: _metrics.on_connect() — test with metrics collector mock
Line 166: _metrics.on_command_sent() — test with metrics collector mock
Lines 175-176: timeout future cleanup when future already removed
Line 202: _metrics.on_disconnect() — test recv_loop disconnect path
Line 209: _dispatch with metrics enabled

exceptions.py (4 lines):

Lines 47-50: TradeError with retcode, symbol, action kwargs

_market_data.py (6 lines):

Lines 200-202: get_full_symbol_info() when symbol not in _full_symbols
Lines 398-399: secondary exception path
Lines 406-407: edge case in symbol resolution

protocol.py (4 lines):

Lines 246, 251, 256, 260: error branches for malformed variable-length fields

_parsers.py (5 lines):

Line 82: timestamp coercion edge case (negative or zero)
Lines 455-457: batch tick parsing edge case
Line 494: copy tick record edge case

_push_handlers.py (3 lines):

Line 200: handler dispatch with no registered handlers
Lines 283-284: book push handler edge case

client.py (2 lines):

Line 272: reconnection metrics callback
Line 301: reconnection success metrics callback

10.2 Symbol cache TTL 

Priority: MEDIUM

Add optional cache invalidation to _MarketDataMixin:

symbol_cache_ttl: float = 0 parameter (0 = no expiry)
Store _symbols_loaded_at: float timestamp
load_symbols() skips refresh if TTL not expired
invalidate_symbol_cache() manual reset method

10.3 Extract currency methods from _market_data.py 

Priority: LOW

Move currency conversion methods to _currency.py mixin (~150 lines):

currency_rate_get()
_resolve_conversion_rates()
_calc_profit_raw()
_calc_margin_raw()

Reduces _market_data.py from 845 to ~695 lines.

Phase 11: Resilience & Observability (v1.0.0)

Goal: Production-ready reliability features.

11.1 Typed push handler callbacks 

Priority: HIGH

Add typed callback registration using events.py dataclasses:

# New typed handler registration
client.on_tick_event(lambda event: ...)  # event: TickEvent
client.on_book_event(lambda event: ...)  # event: BookEvent
client.on_trade_result_event(lambda event: ...)  # event: TradeResultEvent
client.on_account_event(lambda event: ...)  # event: AccountEvent

Implementation:

Add _typed_tick_handlers, _typed_book_handlers, etc. to _PushHandlersMixin
Create event objects from raw push data in handler dispatch
Keep existing untyped handlers for backward compatibility

11.2 Connection health monitoring 

Priority: MEDIUM

Add health check mechanism to transport:

health_check() -> HealthStatus — returns latency, state, last message time
on_health_degraded(callback) — notify when ping latency exceeds threshold
Expose via client.health property
Useful for monitoring dashboards in trading systems

11.3 Graceful degradation on recv_loop errors 

Priority: MEDIUM

Improve error handling when user callbacks raise in _dispatch():

Catch and log callback errors without disconnecting
Add on_callback_error(callback) for error reporting
Prevent a single bad callback from killing the connection

Phase 12: Testing Infrastructure (v1.1.0)

Goal: Comprehensive testing beyond unit tests.

12.1 Integration test framework 

Priority: HIGH

Create tests/test_integration.py gated by PYMT5_INTEGRATION=1:

Bootstrap handshake test (connect, receive bootstrap, disconnect)
Login/logout cycle test
Symbol load test (verify schema parsing against live data)
Tick subscription test (subscribe, receive >= 1 tick, unsubscribe)
Heartbeat round-trip test

Implementation:

pytest.mark.integration marker
conftest.py fixture for test server credentials from env vars
CI job on schedule (not per-commit)
Skip all integration tests if env vars missing

12.2 Performance benchmarks 

Priority: MEDIUM

Create tests/test_benchmarks.py using pytest-benchmark:

SeriesCodec.serialize() throughput
SeriesCodec.parse() throughput
pack_outer() / unpack_outer() throughput
AESCipher.encrypt() / decrypt() throughput
Rate limiter acquire latency under load

12.3 Fuzz testing for protocol parser 

Priority: LOW

Create tests/test_fuzz.py using hypothesis:

Random byte sequences → parse_response_frame() should not crash
Random field values → SeriesCodec.serialize() round-trip
Malformed frames → graceful error, no memory leaks

Phase 13: Documentation & Release (v1.2.0)

Goal: Community-ready documentation and release pipeline.

13.1 Protocol reference documentation 

Priority: HIGH

Create docs/protocol_reference.rst:

Frame format specification (outer header, inner header, body)
Key exchange and AES encryption flow
Session lifecycle diagram (bootstrap -> login -> authenticated -> close)
Command ID catalog with request/response schemas
Field type encoding reference
Tick/book push notification format

13.2 PyPI release automation 

Priority: HIGH

Create .github/workflows/release.yml:

Trigger on git tag v*.*.*
Run full test suite across all platforms
Build sdist and wheel
Publish to PyPI via trusted publisher (OIDC)
Create GitHub Release with changelog
Publish docs to GitHub Pages

13.3 API reference documentation 

Priority: MEDIUM

Generate comprehensive API docs with Sphinx autodoc:

All public methods with docstrings
Usage examples per method category
Error handling guide with exception hierarchy
Configuration reference (rate_limit, reconnect, metrics)

Phase 14: Advanced Features (v1.3.0)

Goal: Production trading enhancements.

14.1 Protocol version tracking 

Priority: HIGH

Track MT5 server build number from bootstrap response:

Extract server build from bootstrap body in transport.py
Add server_build: int property
Log warning for unknown build versions
Expose via client.server_build

14.2 Order manager with tracking 

Priority: MEDIUM

Create pymt5/_order_manager.py:

Track pending and active orders by ID
Automatic state updates from trade push notifications
Order lifecycle events (created, filled, partially_filled, canceled)
Position aggregation per symbol
PnL tracking per position

14.3 Connection pool 

Priority: LOW

Support multiple concurrent MT5 connections:

pool = MT5ConnectionPool(accounts=[
    {"server": "...", "login": 123, "password": "..."},
    {"server": "...", "login": 456, "password": "..."},
])
async with pool:
    await pool.broadcast_subscribe_ticks([symbol_id])

Verification Checklist 

After each phase:

python -m pytest tests/ -v --tb=short – all 830+ tests pass
ruff check pymt5/ tests/ – no lint errors
python -m mypy pymt5/ --ignore-missing-imports – no type errors
cd docs && python -m sphinx . _build/html -W --keep-going – docs build
python -m pytest tests/ -v --cov=pymt5 --cov-report=term-missing – coverage >= 99%

Priority Summary 

Priority	Item	Phase
HIGH	Close coverage to 99.5%	10.1
HIGH	Typed push handler callbacks	11.1
HIGH	Integration test framework	12.1
HIGH	Protocol reference docs	13.1
HIGH	PyPI release automation	13.2
HIGH	Protocol version tracking	14.1
MEDIUM	Symbol cache TTL	10.2
MEDIUM	Connection health monitoring	11.2
MEDIUM	Graceful degradation on callback errors	11.3
MEDIUM	Performance benchmarks	12.2
MEDIUM	API reference docs	13.3
MEDIUM	Order manager	14.2
LOW	Extract _currency.py	10.3
LOW	Fuzz testing	12.3
LOW	Connection pool	14.3