Changelog

[0.4.5] - 2026-10-04

Added

• Changes to users in security.users apply without a restart: a removed user or a changed password stops working at once.
• Protection against password guessing on every server, RTSP included: after 10 failed sign-ins in a row the user is locked out from that address for 5 minutes (security.lockout).
• allowOrigin in every section takes several origins separated by commas; a value that is not an origin is ignored with a warning in the log.

Changed

• The packaged service runs as the forgevis user under systemd isolation, and the default archive is /var/lib/forgevis/recordings. An installation that ran as root stays on root after the upgrade; moving it over is described in the security guide.
• Publishing over WHIP takes the publish permission rather than read.
• /cluster/propose is removed.
• /api/health/ready without credentials answers with the status and version only; the details of the checks go to a caller allowed into the management API.

Fixed

• A user with access to one camera (path) opens it over WebSocket and sees no other camera through it.
• websocket.allowOrigin takes effect: the WebSocket opens only from the pages on the list.
• A camera whose id is not a single directory name (contains /, \ or is ..) is not recorded and fails the configuration check.

[0.4.4] - 2026-10-03

Added

• The reconnect section sets the pauses between connection attempts to a camera: by default one attempt at once, then pauses of 5, 10, 15 and 30 seconds with a random spread; one schedule serves recording, HLS, the restream and viewers.
• /api/archive/{camera_id}/segments takes tz_offset_minutes and dates the days on the viewer's clock.

Changed

• A request for the stream of a camera that does not answer is refused at once between attempts, with the cause and the time of the next attempt.
• The log carries one line per failed connection attempt to a camera, with the cause and the time of the next attempt; a recording that comes back is marked by a line with how long it was down.
• On shutdown the service first stops taking API, playback and stream requests, then closes recording and exits as soon as the segments are closed.
• /api/health is removed: process liveness is /api/health/live, readiness and version are /api/health/ready.
• /api/archive/{camera_id}/segments answers 400 to a window wider than a day or with wrong or inverted bounds, and 503 when the database holding the archive index is unavailable, rather than an empty listing.

Fixed

• A camera that keeps its connection open and stops sending video is reconnected after 5 seconds — for recording, HLS, the RTSP restream and the substream.
• A segment closed when the service stops is visible in the archive.
• A cluster node with the streamer role serves the substream of a camera another node records.
• A segment cut short by losing the camera's stream reaches the archive in the database, and runOnSegmentComplete runs for it.
• A configuration change applied on the fly goes through the same checks as startup; a node with a database and an empty nodeName keeps recording its cameras.

[0.4.3] - 2026-09-30

Added

• The /api/cameras/{id}/snapshot endpoint serves a camera's still image from its snapshot URL in the config or snapuri in the database, and keeps it for 15 seconds for every request to that camera.

Changed

• The recording path is set for the whole node in cameraDefaults.record_path; a configuration that sets record_path on an individual camera fails the startup check.

Fixed

• An on-demand stream, such as a substream, connects for the next viewer even when the previous one left before the connection was up.
• Restarting a stream closes the previous connection to the camera at once.
• PTZ works for cameras added after the node started and for cluster cameras.

[0.4.2] - 2026-09-29

Added

• The variants.m3u8 playlist lists a camera's variants for a quality choice in the player: the source stream, a 720p step and the substream. The step is encoded, and the substream connected, only when a player picks it; master.m3u8 stays the source stream.
• The hls.transcodeMaxConcurrent setting limits how many 720p steps are encoded at once.

Changed

• HLS segment times follow the wall clock, and every segment of every HLS playlist, Low-Latency included, carries EXT-X-PROGRAM-DATE-TIME with the moment it starts.

Fixed

• In HLS, audio and video start together.
• The RTSP restream carries video with the camera's own timestamps, so a stream at any frame rate plays in real time.
• Low-Latency HLS lists every part with its real duration, within the bounds Safari accepts, and a finished segment enters the playlist at once.
• HLS cuts segments and parts by the camera's own time.
• An HLS playlist always spans at least three target segment durations.
• G.711 audio in HLS keeps time with the video.
• The first request for the playlist of a substream or a 720p step waits for its first segment, so the player stays on the variant chosen.

[0.4.1] - 2026-09-24

Added

• A recording names the ForgeVis version that wrote it, in its metadata (©too).

Fixed

• Recordings with AAC audio describe the audio track exactly as ISO/IEC 14496-1 defines it, and a progressive MP4 states the track's bitrate.

[0.4.0] - 2026-09-11

Added

• forgevis --retention deletes recordings older than retention_days — set in cameraDefaults and, where a camera differs, on the camera itself. With no value, and with retention_days: 0, recordings are kept indefinitely. Recordings of cameras the configuration no longer has are kept for the cameraDefaults value.
• forgevis --retention --dry-run reports what would be deleted, camera by camera: how many recordings go, how many stay, and the archive depth each camera ends with.
• A camera read from the database, and one handed over by the cluster, carries its archive depth: the value set on the camera reaches its settings on whichever node records it.
• Retention runs on a node with a database and inside a cluster too: a node deletes only the recordings it wrote itself, and a file's row in the recordings table goes with the file.

Changed

• A recording error's type names the cause: connect for the network or the camera, stream for a camera that answered and went quiet, write for this node's disk, internal for a fault in the service. An error the service does not recognise is labelled unclassified and written to the log in full.
• The segment listing answers with an object: archive holds the segments of the requested window, days every day the camera has recordings for. A calendar and a timeline are built from one request.
• Without from and to the segment listing covers today.
• A segment that began before the window and runs into it is part of the listing.

Fixed

• A camera's metrics go when the camera does: the last-frame, reconnect and recording-error series are dropped as it leaves the configuration.
• A connect timeout for a camera is written to the log once.
• Export and timelapse end at the end of the selected window, even when the recording has a gap inside it.
• A stuck ffmpeg in an export or timelapse is stopped by the api.archiveJobTimeoutSec timeout (one hour by default); the job fails without holding up the ones after it.
• Finished export files are kept in api.archiveExportDir (/var/lib/forgevis/exports by default), accessible to the service alone, and are removed an hour after they are ready — including ones left from a previous run. The parts an assembling node downloads from its peers are gathered there too.
• With mp4 recording, export and timelapse skip the segment still being written — it cannot be read until it closes. With fmp4 it is included up to its last written fragment.

[0.3.10] - 2026-09-10

Fixed

• Cameras that send the SSRC in a non-standard form in the SETUP response (e.g. Rubetek) connect and record.

[0.3.9] - 2026-08-29

Fixed

• Audio samples in fragmented recordings are marked as independently decodable, matching how third-party tools actually decode them.

[0.3.8] - 2026-08-28

Fixed

• A time-lapse covers the window that was asked for: the range is cut before the footage is sped up.
• An export or time-lapse job that encodes no frames ends as a failure that says why.

[0.3.7] - 2026-08-27

Changed

• A hot reload reports its result in one line: how many camera tasks were started, stopped, removed, left unchanged, and are running. Per-camera lines are written only for cameras that changed.

[0.3.6] - 2026-08-27

Added

• HTTPS for every HTTP listener on the node — the management API, HLS, WebRTC signalling, archive playback and metrics. Turned on through the tls section: one certificate for the node and one switch for every listener at once, because a browser that loaded the interface over https will not fetch an HLS playlist over http. Credentials stop crossing the wire in the clear; whether a client can also verify who it is talking to depends on whether it knows the issuing authority. tls.enabled: true without a readable cert and key fails the startup. The certificate is read at startup, so replacing it needs a restart.

Changed

• TLS is configured once for the whole node: the encryption, serverCert and serverKey keys inside the api and webrtc sections are replaced by a top-level tls section.
• The cluster certificate documentation now covers how long a certificate lasts and how to issue one for an installation without the platform.

[0.3.5] - 2026-08-26

Added

• Archive playback server: a request names a camera and a moment, and the recording starts there. The node finds the frame through the index every file keeps in its header, so a player never has to parse the file or work out byte offsets for itself. Playback begins at the nearest key frame, so the first seconds after a seek hold together, and a moment with nothing recorded is answered with the next recording there is. A browser is served a plain file (format=mp4), tools are served the fragments. A listener of its own, off by default — see the playback section of the configuration.
• A node can leave a cluster from any node, not only from the leader. Membership is changed by the leader, and the departing node asks it over the consensus channel with the certificate it was issued. Only the departing node's own id travels: the leader works out the new voter set from its own membership, because the leaver's view may be behind.
• Demoting the current leader out of the voter set reaches a new election. The remaining voters are asked to elect over the consensus channel, where nodes present certificates. The prompt is needed because a demoted leader stays a learner and keeps sending heartbeats: on its own, the others' election timers never expire.
• A cluster-state request that reaches a node other than the leader is answered with a redirect to the leader: the caller repeats it with its own credentials. Nodes reach one another over the consensus channel alone, where they present certificates.
• A node reports its own name on /api/metrics. The platform reads it there when a server is registered rather than taking it from a form: cameras are handed to a recorder by name, and a mismatch of one character raises no error anywhere — the node reads an empty camera list and goes on reporting itself healthy.
• The platform owns the database structure: a node creates nothing in it and needs no rights to change the schema — reading cameras and writing recordings is the whole grant. The trigger that announces camera changes is installed with the schema. For an installation with its own database and no platform, the schema is published as a plain SQL file.
• The recording in progress is available in the archive listing: a node answers which file it is writing for a camera and from what moment, so a timeline shows the last few minutes as well as the closed segments. Requires record.format: fmp4 — a progressive recording cannot be played before it is closed.
• Cluster nodes authenticate each other with certificates. The Control Plane issues them: a node creates its own private key and a signing request, and only the request leaves the node. Until a certificate arrives the consensus port stays closed, which the node reports in its log.

Changed

• POST /cluster/init requires cluster_id, and it has to be a UUID. That identity is what a camera row points at when it names the cluster that owns it, and the cluster reads exactly the rows carrying it. A cluster without one would read every enabled camera in the database as its own, and it cannot be re-initialised to acquire one later — so a request without cluster_id is refused. A value that is not a UUID is refused at the same point: cameras.cluster_id is a uuid column, and otherwise the refusal would surface later and look like an unreachable database.
• A camera with recording switched off does not build a recorder: the journal carries an INFO line saying the camera is streamed but not recorded. The level matches what happened — that configuration is a deliberate one, not a fault.
• The node name is set by the top-level nodeName setting rather than inside the cluster section: a single node with a database needs it too, and there is no cluster there at all. An empty value means the system hostname. There is no built-in default — a shared one would have two nodes report the same name and record the same cameras; with a database configured, a node that can resolve no name refuses to start. A name too long is refused at startup as well: the database matches it against a 64-character column. The node logs the name it resolved and where it came from.
• The consensus channel is always protected; cluster.tls.enabled and cluster.tls.require_client_auth have been removed. Paths to your own certificates are still set in cluster.tls.
• A management request that reaches a node other than the leader is answered with a redirect to the leader: the caller repeats it with its own credentials, so nodes need no credentials for one another.
• Node liveness travels over the consensus channel, so cluster state does not depend on whether the management API is behind a password.
• An invalid cluster.rpc_addr is rejected at startup.
• The segment listing at /api/archive/{camera}/segments includes the recording in progress, marked with an is_open field, so one request returns a camera's whole archive up to the current minute. Every other segment carries is_open: false.
• The Raft state directory is set in cluster.dataDir; it defaults to /var/lib/forgevis/data. The path is absolute, so it does not depend on the directory the service is started from. The node's certificates live there too, under tls. The default matches the directory a running installation already uses, so an upgrade need not set the key.
• A node configured for a cluster warns at startup when its state directory is empty.
• The shipped configuration sets record.format to fmp4. The playback server and the recording-in-progress endpoint work only with it: a progressive mp4 keeps its header at the end of the file and cannot be played until it closes.
• A camera identifier in an archive request holds letters, digits, _ and - and nothing else; anything else is refused with 400.
• An archive request for a camera in a cluster is answered with a redirect to the node recording it when it arrives anywhere else. /api/archive/{camera}/segments and /api/archive/{camera}/current therefore answer the same way on any node of the cluster, and no particular node has to be addressed.

Fixed

• A cluster keeps recording when a node cannot read the camera list from the database: an unavailable list is not taken for an empty one, and assignments stay in place until a read succeeds.
• A failed camera read does not stop the cameras on a single node: the reload is skipped and the current camera set is kept.
• A node removed from a cluster can be added back without restarting the service: leaving erases its local cluster state right away, and the node starts recording the cameras assigned to it as soon as it is part of a cluster again.
• A failure of the cluster machinery is detected automatically: the node reports itself as not ready and recovers on its own.
• Removing a node that is no longer part of the cluster succeeds.
• A camera that names a specific cluster node is recorded by that node, including when the camera list comes from the database.
• The hls.allowOrigin setting is applied: a page reaches HLS from an address the configuration allows.

[0.3.4] - 2026-08-13

Fixed

• Fixed seeking in fMP4 recordings: fragments are cut at keyframes, so jumping to a position lands on a frame playback can actually start from.
• Fixed the audio track duration in MP4 recordings.
• Fixed the H.265 profile and level advertised in recordings: it did not match the stream, which could make strict decoders and in-browser playback refuse the file.
• Recordings now start on a keyframe, so playing a file from the beginning no longer shows artefacts for the first second.
• A segment interrupted by a lost camera connection or by a stalled stream is now closed properly: in MP4 format such a file previously would not open at all.
• Frame durations in recordings follow the camera's actual stream: archive time no longer drifts from real time (about 80 seconds a day on a camera running at 25.02 fps), and dropped frames no longer collapse the timeline.
• A camera clock restart no longer corrupts a recording: the segment is closed and the next one starts on a fresh timeline.
• A segment's start time in the database now matches its first recorded frame, so time-range export lands on file boundaries more accurately.
• Added timeouts to camera connections and internal network calls: an unreachable camera or node no longer holds a task until the system TCP timeout expires.
• Fixed a duplicate camera connection when several consumers subscribed at the same time — the connection stays single.
• An RTSP restream client that stops reading no longer holds the camera connection open.

Changed

• An oversized record.segmentDuration is now rejected at startup: the segment index would not fit in the file, silently disabling time-based seeking.

[0.3.3] - 2026-08-05

Fixed

• Fixed camera settings (recording, audio, sub-stream) not being applied when a camera moves to another cluster node.

[0.3.2] - 2026-08-04

Fixed

• Improved cluster state storage resilience to sudden power loss.
• Fixed audio/video synchronization in archive recordings; audio frame durations now account for the codec (AAC, G.711, Opus).
• Fixed RTSP restreaming for cameras with G.711 and Opus audio.
• Improved node address registration in the cluster at startup; node status changes no longer reset node details.

[0.3.1] - 2026-08-04

Changed

• Cluster State Sync Streamlined: Recorder liveness and address metadata now travels through a single Raft state entry per node instead of six separate ones. After a node has been running for a minute, its recurring heartbeat payload shrinks by roughly 6×, and Raft snapshots carry fewer keys per member. No externally-visible behaviour change — the reduction is entirely in background inter-node traffic. Most noticeable on deployments with many recorders.

[0.3.0] - 2026-06-03

Added

• Cross-Node Archive Export: When a camera moved between recorder nodes during the requested export window, the resulting MP4 is now seamlessly stitched together from fragments living on different recorders. The platform/backend orchestrates: it reads the recordings table, partitions the requested window into per-node runs, submits each run as a stream-copy export to its owning recorder, then delegates the final concat to one chosen recorder ("assembler") to keep heavy I/O off the backend host. The final MP4 streams to the client through the backend as a thin proxy. Canonical migration patterns are supported — single migration, multi-hop, even repeat-visits to the same node (e.g. S1 → S2 → S3 → S2 becomes one playable MP4 with no internal gaps). See Multi-node archive guide.
• Cross-Node Time-Lapse: Same fan-out as plain export, with a final setpts=PTS/N pass on the assembler. Speed factor is an integer in [2, 120]; output duration ≈ raw_duration / speed.
• Resilient Job State: Multi-node export jobs survive recorder restarts (each sub-export is resubmitted with a fresh id on 404, up to 3 attempts per run) and backend restarts (in-flight jobs in pending/exporting/assembling resume their orchestrator tasks on the next backend startup and pick up at exactly the recorded state).
• DELETE /api/archive/jobs/{id}: Lets operators (and the backend orchestrator on cleanup) explicitly evict a finished archive MP4 from a recorder rather than waiting for its 1-hour TTL sweep. Returns 204 on success, 404 if the job is already gone.

Changed

• recordings.node_id Renamed to recordings.node_name: The column that names the recorder behind a segment now stores a human-readable identifier (config.cluster.node_name, e.g. S1) instead of the numeric Raft id. This makes joining recordings to the platform servers registry a single column equality, with no live-cluster query required. The Alembic migration runs automatically on backend upgrade.

0.2.x

[0.2.11] - 2026-05-31

Changed

• Database Connection Pool Sized Up: The Postgres backend's max connection count was raised from 5 to 10. Old value queued metadata inserts behind any long-running query (e.g. an archive scan) on a busy node.

Fixed

• Cluster State Events Drop Under Bursts: The internal channel that carries cluster state-change events (camera assignments, role updates, storage transitions) had a 100-event capacity. During a burst — e.g. mass camera reassignment after a node demotion — the slowest subscriber could fall behind and silently miss events. Capacity is now 1000.
• Corrupted Snapshot Index Silently Masked: When applying a snapshot from the leader, an unparseable snapshot_id field used to default to 0, which looked indistinguishable from a freshly initialised store and could mask state divergence between leader and followers. The bad value is now logged loudly and the existing snapshot index is preserved until the next valid snapshot arrives.
• Less Cryptic RocksDB Recovery Errors: If the embedded key-value store can't be opened (path not writable, locked by another process, corrupted column family), the error message now points the operator at the specific cause instead of a generic "failed to open" or "cf not found".

Added

• Strict Configuration Validation at Startup: The server now refuses to start if config.yaml contains any of the following problems, aggregating every issue into a single error message instead of failing on the first one. Catches misconfigurations at boot rather than at first request / first recording / first encrypted handshake.
  • Two services bound to the same port (e.g. RTSP and API both on 8554).
  • A camera with record: true but no resolvable record_path (neither on the camera itself nor in cameraDefaults.record_path).
  • A camera source URL that is empty, lacks rtsp:// / rtsps:// scheme, or fails URL parsing. Credentials are masked in error messages.
  • encryption: true on the API or WebRTC server with missing or unreadable serverCert / serverKey files.

[0.2.10] - 2026-05-30

Fixed

• Hot-Reload Robustness: A configuration reload triggered while an editor is mid-write (yielding a truncated YAML file) is now retried up to three times with 300ms between attempts before being abandoned. Previously a transient parse error silently kept the old config active without explanation.
• Database Schema Auto-Setup: When database.enabled=true, the schema/notification trigger is now installed at startup (previously a commented-out path). If the cameras table does not yet exist, the failure is logged as a warning instead of being silently skipped — hot-reload via DB notification only works after the schema is in place.
• Archive Query Bound: Database-backed archive segment lookups (driven by the database: backend) now cap results at 10,000 rows per request and log a warning when the cap is hit. Guards against accidental multi-month time ranges pulling huge result sets over a small connection pool.
• ONVIF SOAP Credentials Escaping: Usernames and topic-filter strings in SOAP requests are now XML-escaped. A password or username containing &, <, or " no longer corrupts the WS-Security envelope.
• ONVIF Response Hardening: SOAP responses are now read with a 10 MiB hard cap and rejected if they contain inline <!DOCTYPE or <!ENTITY declarations. Defends against a compromised camera attempting to exhaust memory or probe for XXE.
• WebRTC SDP Size Limit: WHEP/WHIP offer bodies larger than 128 KiB are rejected with a clear error. Real-world SDPs are an order of magnitude smaller; this stops malicious or malformed clients from streaming huge bodies.
• Frame Timestamp Hardening: An edge case in the per-frame timestamp normalization could theoretically cause the recorder to abort if an audio frame arrived before any video frame in the same session. The path was unreachable in practice but the fix removes the latent risk.

[0.2.9] - 2026-05-30

Added

• Per-Camera Health Metrics: New Prometheus series for alerting on stale or unstable cameras. camera_last_frame_timestamp_seconds{camera="..."} is updated on every video frame, so a Grafana alert on time() - camera_last_frame_timestamp_seconds > N fires when a camera goes quiet. camera_reconnects_total{camera="..."} counts stream-reconnect attempts per camera. Existing recording_errors_total now carries a camera label in addition to type.

Changed

• Graceful Shutdown — Full Coverage: On SIGTERM / SIGINT, all background services (recorders, RTSP, WebRTC, management API, metrics server) are now stopped cleanly. Recorders get a 3-second drain window to finalize in-flight segments; the whole sequence is capped at 15 seconds so systemd never has to escalate to SIGKILL. Previously only camera tasks and HLS server were signalled.

Fixed

• WebRTC Session Leak: WHEP and WHIP sessions are now evicted after 30 minutes of inactivity (no ICE candidate exchange, no state transitions). A background sweep runs every 60 seconds, closes the underlying peer connection, and removes the entry. Prior to this fix, abandoned sessions (client crashed, network dropped before DELETE) accumulated in memory for the process lifetime.

[0.2.8] - 2026-05-30

Added

• Async Archive Operations (Export & Time-Lapse, unified job API): POST /api/archive/{camera}/export?from&to and POST /api/archive/{camera}/timelapse?from&to&speed both return 202 Accepted with a job_id. Job lifecycle is observed via GET /api/archive/jobs/{job_id} (pending → running → done/failed). Result is fetched via GET /api/archive/jobs/{job_id}/download (inline by default; ?download=1 forces attachment). Time range capped at 4 hours; completed jobs expire after 1 hour. Concurrency: 4 parallel exports (stream-copy) and 2 parallel time-lapses (re-encode) per node.
• Periodic Camera Hook: New record.runPeriodically block lets operators run an arbitrary script per camera at a fixed interval (default 60s). A global concurrency limit (maxConcurrent, default 50) bounds CPU spend across all cameras; cameras stagger their first tick at startup to avoid stampedes. Designed for live-preview thumbnail generation via ffmpeg without blocking the recorder.
• Health Probes: New /api/health/live (process up) and /api/health/ready (all dependencies reachable: storage, cluster store, recordings directory, license). Existing /api/health retained for backward compatibility.
• API Rate Limiting: Per-IP token-bucket throttling on the management API. Configurable via api.rateLimit (enabled, perMinute, burst); defaults to 600 req/min, burst 60. Returns 429 Too Many Requests on excess.
• Panic-to-File Crash Dumps: On process panic, a backtrace is written to <logging.directory>/crashes/<unix_ts>_<thread>.log for post-mortem analysis. Stderr/journald output is preserved.

Known Limitations

• All archive operations (/archive/..., runPeriodically) work only with files local to the node. In clustered deployments, if a camera was hosted on another node previously, those segments are not yet accessible through the current node's API. Tracked as future work.

[0.2.7] - 2026-05-28

Added

• Archive Byte-Range Seek: Recorded fragmented MP4 archives now embed a segment index, enabling fast in-file seeking via HTTP byte ranges in players and analysis tools.
• Argon2id Password Hashing: security.users[].pass accepts a new argon2: prefix (PHC format), recommended for production deployments. Comparison runs in constant time.

Changed

• HLS Master Playlist Bandwidth: BANDWIDTH is now derived from real segment sizes instead of a static estimate, giving adaptive bitrate clients accurate input.
• HLS Media Playlist Timing: Each segment's EXTINF reports its actual duration, and TARGETDURATION adapts to the observed maximum, reducing player stalls on variable-duration segments.
• HLS Low-Latency Compatibility: Parts at segment boundaries now declare INDEPENDENT=YES so Safari and other strict players can join a live stream mid-segment.
• Credential Format Tightening: security.users[].pass now requires an explicit prefix (argon2:, sha256:, plain:). Bare values without a prefix are rejected (fail-closed) to prevent typos from silently demoting a hashed credential to plaintext.

Fixed

• ONVIF PTZ Connection Reuse: PTZ now inherits endpoint, port, and credentials from the camera's RTSP source when no explicit ONVIF overrides are configured, including non-default ports.
• Large MP4 Recordings: Recordings larger than 4 GiB no longer silently truncate internal chunk offsets — files now grow past the 32-bit limit correctly.
• Audio Tail at Session End: Audio RTP frames are no longer dropped when the video track ends slightly earlier in the same streaming session.
• Standalone License Camera Limit: An empty or missing camera limit no longer behaves as unlimited; the unlicensed default is applied instead.

Breaking Changes

• Password Prefix Required: security.users[].pass must use one of argon2:, sha256:, plain:. Migration: add plain: to existing unprefixed values, or rehash to argon2: / sha256:.

[0.2.6] - 2026-04-10

Added

• WebRTC Session Management API: Added /api/webrtc endpoints to list active WHEP sessions, inspect a session by ID, and force-close sessions from the management API.
• Built-in WebRTC Test Pages: Added packaged HTML diagnostics pages (webrtc-test.html, webrtc-test-ptz.html) for quick browser-side validation of WebRTC playback and PTZ control.

Changed

• WHEP/WHIP Signaling Routes: WebRTC signaling routes were aligned with standardized endpoint style (/{camera_id}/whep, /{camera_id}/whip) and now include OPTIONS/PATCH handling and ICE Link headers for clients.
• Frontend Static HTML Serving: Management API now serves top-level *.html pages from packaged/dev HTML locations with filename safety validation.

[0.2.5] - 2026-04-05

Added

• Native WebRTC Playback (WHEP): Added built-in WebRTC server with a WHEP endpoint and session lifecycle handling (POST offer + DELETE session close).
• WebRTC Audio Codec Coverage: Added audio delivery for AAC, G.711 A-law (PCMA), and G.711 μ-law (PCMU) in WebRTC playback.
• PTZ in Main API: Added ONVIF PTZ control endpoints in the main ForgeVis API (/api/ptz/move, /api/ptz/stop) with capability checks.

Changed

• Control Plane Camera Contract: Extended camera model/DTO fields in Control Plane to include stream behavior flags and node assignment input.

Fixed

• ONVIF Credentials Source for PTZ: PTZ ONVIF access now derives endpoint/auth data from RTSP camera source consistently.

[0.2.4] - 2026-03-20

Added

• Standalone Recorder Watchdog: Added a periodic supervisor that detects unexpectedly stopped recorder tasks and restarts them automatically in standalone mode.

Changed

• Cluster Health API Consolidation: Cluster node health, roles, and storage status are now provided via /cluster/metrics.
• Control Plane Compatibility: Cluster and license views were updated to consume the consolidated cluster metrics response.

Fixed

• Standalone License Camera Count: Corrected licensed camera counting logic in standalone deployments.
• Recorder Recovery Reliability: Reduced cases where recording remained stopped after long upstream outages by restarting dead recorder tasks on watchdog interval.

Breaking Changes

• Removed Endpoint: GET /cluster/status was removed. Use GET /cluster/metrics instead.

[0.2.3] - 2026-03-13

Fixed

• Hot-Reload Audio Enablement: Changing a camera from audio: false to audio: true now reapplies stream settings without deleting and re-creating the camera configuration.
• Live Streaming Consistency on Reload: Stream parameter updates now restart active live stream sessions at the hub layer, including non-recording cameras used only for HLS/RTSP delivery.
• RTCP Resilience: Improved handling of malformed short RTCP packets from unstable camera firmware to reduce unnecessary stream interruptions.

[0.2.2] - 2026-03-08

Added

• Storage-Aware Cluster Failover: Nodes now report disk usage and automatically switch to Draining on low free space (<=8%), returning to Normal after recovery (>=18%).
• Manual Drain/Resume Controls: Added API and dashboard actions for forcing Draining/Normal with manual override behavior.
• Extended Cluster Metrics API: /cluster/metrics now includes per-node storage usage and storage lifecycle state.
• CentOS 10 Packaging Support: Added build support for CentOS/RHEL 10 packaging pipeline.
• TLS for Inter-Node Cluster Traffic: Added TLS protection for internal communication between cluster nodes.

Changed

• Cluster Placement Policy: Scheduler excludes Draining recorder nodes from new camera assignments and rebalances to available healthy nodes.
• Cluster Topology UI: Node cards now show lifecycle badge (Online/Draining/Offline), storage usage progress bar, and a draining node summary.
• Metrics Architecture: Refactored metrics into modular collector/registry/server components, and moved cluster_nodes_* updates to on-demand evaluation during /metrics scrapes.

Fixed

• Leader Election on Demotion: Improved leader transition behavior when demote operations occur.
• Offline Safety Checks: Cluster API now rejects role and storage-state updates for offline nodes.

[0.2.1] - 2026-03-04

Added

• Cluster Node Roles: Added role-based node behavior (recorder / streamer) with runtime updates through cluster API and visibility in cluster status.
• Credential Hashing for Internal Auth: Added support for sha256:-prefixed credentials in security.users (user and pass).
• Cluster Stream Placement Controls: Added role-aware stream handling with per-camera cluster assignment and protocol-specific serving decisions.

Changed

• Cluster HLS/RTSP Policy Propagation: Scheduler and runtime now consistently apply per-camera hls, rtsp, and alwaysRemux settings in cluster mode.
• HLS Behavior Controls: Clarified precedence between global and per-camera remux options and enforced camera-level HLS disable rules.

Fixed

• Recorder Restart Reliability: Improved restart behavior after runtime camera state changes and reassignment cycles.
• License Reload Safety: Added stronger validation during hot-reload to prevent invalid license state activation.

[0.2.0] - 2026-01-10

Added

• High Availability Clustering: Completely new distributed architecture based on the Raft consensus algorithm with an embedded key-value store for replicated state.
  • Shared State: All nodes maintain a consistent state of camera assignments and node health.
  • Automated Failover: Leader automatically detects dead nodes (heartbeat timeout > 15s) and redistributes their cameras to healthy nodes.
  • Internal Proxying: API requests sent to Follower nodes are transparently proxied to the Leader using internal HTTP client.
  • Auto-Join: New nodes with empty state and configured peers automatically discover and join the cluster as Learners.
  • Zero-Downtime Migration: Cameras can be moved between nodes dynamically via API.
• Cluster Monitoring:
  • New /cluster/status endpoint showing real-time health (Online/Offline) of all nodes.
  • Integration with Frontend Dashboard to visualize cluster health.
• Storage Backend:
  • Embedded key-value store: High-performance local persistence for Raft log and state machine snapshots.
  • Compaction: Automatic log purging and snapshotting to prevent disk overflow.

0.1.x — earlier releases

[0.1.16] - 2026-02-21

Fixed

• Progressive MP4 Playback: Improved standard MP4 segment structure so playback starts immediately without waiting for full file download.
• fMP4 Atom Compatibility: Aligned fMP4 atom layout and fragment headers with ffmpeg-style output (ftyp, moov ordering, tfhd/trun flags) for better player compatibility.

Improved

• fMP4 Validation Accuracy: Refined box-level verification workflow to compare atom order and key fragment metadata against ffprobe reference output.

[0.1.15] - 2026-02-20

Added

• Dual Recording Format: Added configurable recording format selection via record.format with support for both fragmented MP4 (fMP4) and standard MP4.
• Standard MP4 Recording: Added standard MP4 output mode for archive recording with support for H.264/H.265 video and AAC/G.711 audio tracks.

Changed

• Recorder Architecture: Unified recorder implementation into module-based writers for cleaner format handling and easier extension.

[0.1.14] - 2026-02-20

Added

• Fast Seeking: Implemented mfra (Movie Fragment Random Access) atom at the end of MP4 files for instant seeking in supported players (VLC, QuickTime, Web).

Optimized

• File Finalization: Improved file closing process to correctly append the Random Access Table.

[0.1.13] - 2026-02-20

Fixed

• Seeking & Progressive Playback: Fixed FMP4 structure (trun and tfhd atoms) to correctly support seeking and progressive playback.
• MP4 Compatibility: Changed brand from iso5 to isom for broader player compatibility.
• File Duration: Fixed duration update (mvhd atom) even when recording terminates unexpectedly.

[0.1.12] - 2026-02-14

Optimized

• Performance: Disabled high-frequency metrics to reduce lock contention under high load (>1000 cameras).
• Disk I/O: Optimized MP4 header updates to reduce disk I/O operations during recording.

[0.1.11] - 2026-02-10

Fixed

• HLS Stability: Fixed infinite playlist restart loop when upstream RTSP stream reconnects.
• Logging: Reduced log verbosity for cleaner production logs.

[0.1.10] - 2026-02-05

Fixed

• Audio Support: Fixed G.711 (PCMU/PCMA) audio support by updating the RTSP client library.
• Error Handling: Improved error reporting for RTSP connection failures.

[0.1.9] - 2026-01-28

Added

• Audio Codecs: Added support for G.711u (PCMU) and G.711a (PCMA) codecs.

Fixed

• Video Synchronization: Fixed A/V desync issues on streams with unstable timestamps.

[0.1.8] - 2026-01-15

Fixed

• Timestamp Handling: Improved handling of non-monotonic timestamps from certain camera models.

[0.1.7] - 2025-12-30

Added

• Stream Security (Internal Auth): New security config block with internal user definitions, IP/CIDR filters and per-action permissions (publish, read, api, metrics) for unified access control across the system.
• Protected Live Streams & Metrics: When security.users is not empty, all live-read endpoints (RTSP, HLS, WebSocket) now require read permission and the /metrics endpoint requires metrics permission, with support for anonymous user: any and IP-based exemptions (for example, local Prometheus scrapers).

Changed

• Backwards-Compatible Defaults: If the security block is omitted or the users list is empty, all endpoints continue to work without authentication, preserving behavior from previous versions.

[0.1.6] - 2025-12-28

Added

• Per-process Disk I/O Metrics (Linux): New Prometheus metrics sourced from /proc/self/io to track read/write bytes per ForgeVis process: process_io_read_bytes_total, process_io_write_bytes_total, process_io_cancelled_write_bytes_total.
• Grafana I/O Panel: Updated default Grafana dashboard with a new "Process Disk IO (Read/Write MB/s)" panel for visualizing per-process disk throughput and correlating it with Frames Lost and broadcast lag.

[0.1.5] - 2025-12-24

Added

• Low-Latency HLS (LL-HLS): Native Low-Latency HLS mode with fMP4 segments and HLS parts. New hls.variant: "lowLatency" with configurable segmentDuration and partDuration, optimized for hls.js lowLatencyMode.

Fixed

• B-Frame Support: Implemented B-frame support in fMP4. Added ctts (Composition Time to Sample) atom and frame reordering logic (reordering buffer) to correctly calculate Presentation Time Stamp (PTS) and Decode Time Stamp (DTS). This eliminates video stuttering on streams with B-frames.
• Synchronization (tfdt drift): Fixed time drift in tfdt atom. Fragment decode base time is now derived from the actual DTS of the first frame instead of accumulated durations, preventing desync during long recordings.

[0.1.4] - 2025-12-24

Fixed

• Archive Recording: Fixed issue with zero duration in mvhd header of fMP4 files. Players now correctly display duration and support seeking.
• Synchronization: Fixed hardcoded frame duration issue. Duration is now calculated dynamically based on timestamps, eliminating A/V desync.
• Performance: Moved file metadata updates to a background process to prevent delays between segment recordings.

[0.1.3] - 2025-12-23

Fixed

• Archive Seeking: Critical fix for fMP4 trun atom flags and per-sample flags. Browsers can now correctly seek to keyframes within fragments.
• WebSocket Stability: Fixed infinite reconnection loops in the React player when switching between Live and Archive modes.

[0.1.2] - 2025-12-22

Added

• Frontend Redesign: Migrated UI components (CameraList, ArchiveList) to Bootstrap 5 for a cleaner, responsive look.
• Player Improvements:
  • Improved fullscreen behavior with aspect ratio preservation.
  • Custom timeline control.

Fixed

• Archive Seeking: Fixed precise seeking in archive playback.

[0.1.1] - 2025-12-21

Added support for Low-Latency Streaming via WebSocket (MSE).

Streaming

• Low-Latency MSE (WebSocket)
  • Direct streaming of fMP4 fragments to browser via WebSocket
  • Ultra-low latency (< 1 second)
  • Media Source Extensions (MSE) support

[0.1.0] - 2025-12-20

Major release introducing Management API, Graceful Shutdown, and Legacy System Integration.

Management API

• HTTP Control Interface
  • New Management API server running on port 9997 (configurable)
  • /health endpoint for system monitoring
  • /cameras/start endpoint to bulk start all configured cameras
  • /cameras/stop endpoint to bulk stop all cameras
  • CORS support for web integration

Reliability

• Graceful Shutdown
  • Proper handling of SIGINT (Ctrl+C) and SIGTERM signals
  • Ensures all active recordings are finalized and saved to disk before exit
  • Clean shutdown of RTSP and HLS servers

[0.0.10] - 2025-12-19

Added support for ONVIF events integration.

ONVIF Integration

• Event Monitoring
  • Subscribe to camera events via ONVIF PullPoint
  • Support for Motion Detection, Digital Inputs, Line Crossing, and more
  • Configurable polling interval and event topics
  • Automatic credential extraction from RTSP URL
  • Per-camera ONVIF configuration in camera.yaml

[0.0.9] - 2025-12-16

HLS compatibility fixes for working with hls.js default settings.

HLS Improvements

• Dynamic Playlist Parameters
  • #EXT-X-TARGETDURATION now derived from segmentDuration config
  • #EXTINF uses config value instead of hardcoded duration
  • Removed #EXT-X-START tag for compatibility with default hls.js settings
  • Support for configurable segment durations (1s, 4s, etc.)

Bug Fixes

• Playlist Compatibility
  • Fixed playlist generation to work with hls.js without custom configuration
  • Playlists now match ffmpeg format
  • Removed tags causing issues with default hls.js configuration

Configuration

• hls.segmentDuration parameter now affects:
  • #EXT-X-TARGETDURATION in playlist
  • #EXTINF for each segment
  • Actual duration of created segments

[0.0.8] - 2025-12-12

Production monitoring and metrics system with Grafana dashboard.

Monitoring & Observability

• Prometheus Metrics

  • 14 aggregate metrics for system monitoring (scales to 2000+ cameras)
  • HTTP /metrics endpoint (default port 9998)
  • Automatic hostname labeling for multi-server deployments
  • Camera status: registered, recording, errors
  • Stream metrics: active connections (main/sub streams)
  • Performance: video frame rate, broadcast buffer status
  • HLS metrics: active muxers, segment size
  • Error tracking: recording errors by type, broadcast lag events
  • System metrics: process memory (RSS)
  • CORS support for cross-origin access

• File Logging

  • Daily log rotation with tracing-appender
  • Separate files: forgevis-info-YYYY-MM-DD.log, forgevis-error-YYYY-MM-DD.log
  • Configurable log levels (trace, debug, info, warn, error)
  • Console output with proper filtering
  • Cleaned up duplicate logs from internal components

• Grafana Dashboard

  • Pre-built JSON dashboard with 8 visualization panels
  • 4-row layout: Camera Status, Performance, Resources, Errors
  • Multi-server support with hostname filter dropdown
  • 5-second auto-refresh for real-time monitoring
  • Memory usage in MB (proper unit conversion)
  • Complete metrics reference in documentation

Bug Fixes

• HLS Muxer Metrics

  • Fixed HLS muxer counter not decrementing on cleanup
  • Implemented proper lifecycle tracking with Drop trait
  • Added shutdown channel to stop tasks cleanly
  • Prevented duplicate metric decrements with coordination flag
  • Ensures accurate count of active HLS muxers

• Memory Management

  • HLS muxer tasks now properly terminate on cleanup
  • Fixed memory leak from orphaned background tasks
  • Shutdown signal propagates to all muxer components

Documentation

• Monitoring Guide (English & Russian)
  • Complete Prometheus setup instructions
  • Grafana dashboard import guide
  • All metrics with descriptions
  • Multi-server deployment patterns
  • Troubleshooting memory and performance

Configuration

• New Settings

  logging:
    directory: /var/log/forgevis
    level: info
    console: true
  
  metrics:
    enabled: true
    address: :9998
    allowOrigin: '*'

[0.0.7] - 2025-12-11

HLS playlist naming convention changed for better API consistency.

HLS Improvements

• Playlist Naming
  • Video playlist: video.m3u8 → media_0.m3u8
  • Audio playlist: audio.m3u8 → media_1.m3u8
  • Video init segment: init_video.mp4 → init_media_0.mp4
  • Audio init segment: init_audio.mp4 → init_media_1.mp4
  • Video segments: video_{N}.m4s → seg_media_0_{N}.m4s
  • Audio segments: audio_{N}.m4s → seg_media_1_{N}.m4s
  • Centralized HLS naming constants in a single internal module

[0.0.6] - 2025-12-11

RTSP UDP transport support.

RTSP Improvements

• UDP Transport
  • UDP transport support alongside TCP (interleaved)
  • Configurable UDP ports for RTP/RTCP (udp_rtp_port, udp_rtcp_port)
  • Shared UDP sockets for multiple clients (efficient resource usage)
  • Automatic transport detection from client SETUP request
  • Works with VLC, ffplay, and other RTSP clients

Configuration

• New RTSP Settings
  • rtsp.udp_rtp_port (default: 8000) — UDP port for RTP packets
  • rtsp.udp_rtcp_port (default: 8001) — UDP port for RTCP packets

[0.0.5] - 2025-12-09

AAC audio support for recording and streaming.

Video Codecs

• H.265/HEVC Support
  • Full H.265 codec support alongside H.264
  • Automatic codec detection (H.264 or H.265)
  • H.265 RTP packetization (RFC 7798)
  • VPS/SPS/PPS parameter extraction
  • HVCC format recording

Audio Recording

• AAC Audio Codec

  • Automatic AAC audio stream detection via RTSP
  • AudioSpecificConfig parsing (sample rate, channels)
  • Dual-track fMP4 recording (separate video and audio tracks)
  • Perfect audio/video synchronization in recordings

• Audio Configuration

  • Per-camera audio enable/disable (audio: true/false)
  • Global default audio setting
  • Automatic fallback to video-only if audio unavailable

Audio Streaming

• RTSP Audio Restreaming

  • Multi-track RTSP sessions (video + audio)
  • RFC 3640 AAC RTP payload format
  • RTCP Sender Reports for both tracks
  • Session reuse for SETUP requests (VLC compatibility)
  • Works with VLC, ffplay, and other RTSP clients

• HLS Audio Streaming

  • Multi-variant playlist architecture (master playlist)
  • Separate video and audio playlists
  • Individual video/audio segment files (video_N.m4s, audio_N.m4s)
  • Separate initialization segments (init_video.mp4, init_audio.mp4)
  • Standard HLS.js compatibility

Technical Implementation

• Audio Pipeline

  • retina RTSP client with audio depacketization
  • AAC frames broadcast through StreamHub
  • Dual-channel architecture (video_receiver + audio_receiver)
  • Sample duration calculation (1024 samples per AAC frame)

• FMP4 Writer Enhancement

  • Dual-track mdat/moof box writing
  • Separate audio trak in moov box
  • Audio sample table (stbl) with proper timescale
  • AudioSampleEntry (mp4a) with esds descriptor

Code Quality

• Removed unused variables and functions
• Cleaned up dead code (kept public API functions)
• Zero compilation warnings
• Refactored CameraRecorder (removed unnecessary parameters)

[0.0.4] - 2025-12-09

Hot reload and advanced stream management.

Hot Reload System

• Configuration Watcher

  • Automatic monitoring of config.yaml and conf.d/*.yaml files
  • 500ms debouncing for rapid file changes
  • Real-time camera configuration updates without restart

• Dynamic Camera Management

  • Add/remove cameras on-the-fly
  • Update camera sources (URL changes)
  • Enable/disable recording per camera
  • Graceful stream cleanup on camera removal

Stream Management Architecture

• Centralized Camera Registry

  • StreamHub as single source of truth for camera configurations
  • Automatic registration/unregistration during hot reload
  • Support for main stream and substream configurations

• On-Demand Streaming

  • HLS muxers created only when client requests stream
  • RTSP streams for registered cameras only
  • Validation before muxer creation (eliminates unnecessary resource allocation)
  • Proper 404 responses for non-existent cameras

• Stream Lifecycle Management

  • CancellationToken-based stream termination
  • Immediate cleanup when camera is removed from config
  • Automatic stop of both main and substream on camera deletion
  • RTSP client disconnection on stream cancellation

Technical Improvements

• Removed configuration duplication from HLS/RTSP servers
• Unified stream validation through StreamHub
• Better error handling for missing cameras
• Thread-safe camera registry

Fixes

• Fixed RTSP streams continuing after camera deletion
• Fixed HLS muxer creation for non-existent cameras
• Fixed substream (/sub) on-demand creation
• Eliminated all compilation warnings

[0.0.3] - 2025-12-08

Automation and integration through Event Hooks.

Event Hooks

• Segment Event Hooks

  • runOnSegmentCreate - Triggered when segment recording starts
  • runOnSegmentComplete - Triggered when segment recording finishes
  • Environment variables: SF_PATH, SF_SEGMENT_PATH, SF_SEGMENT_DURATION, RTSP_PORT

• Integration Capabilities

  • Discord/Slack/Telegram notifications
  • Cloud storage upload (S3, GCS, Azure)
  • Thumbnail and preview generation
  • Prometheus metrics export
  • Automatic cleanup of old recordings
  • Custom video processing workflows

Technical Implementation

• Asynchronous hook execution without blocking recording
• Success/error logging for hook execution
• Shell command and script support
• Comprehensive documentation with integration examples

[0.0.2] - 2025-12-07

HLS streaming and web interface implementation.

[0.0.1] - 2025-12-04

Initial development release of ForgeVis video recorder and restreaming server.

Recording Features

• Fragmented MP4 Recording

  • Full ISO BMFF (Base Media File Format) implementation
  • fMP4 format with proper box structure (ftyp, moov, moof/mdat)
  • AVCC format for H.264 (4-byte length prefixes)
  • Files playable while recording
  • 2-second fragment intervals for low-latency access

• Segment Management

  • Time-based segmentation (configurable, default 60 seconds)
  • Automatic directory creation
  • Path templates with {camera_id} and {timestamp} placeholders
  • Sequential numbering with 6-digit padding

Streaming & Distribution

• StreamHub Architecture

  • Single RTSP connection per camera
  • Broadcast (fan-out) pattern for distributing frames to multiple consumers
  • 1000-frame buffer for multiple consumers
  • Automatic SPS/PPS extraction
  • Transparent frame distribution to recorder and restream server

• RTSP Restreaming Server

  • Full RTSP protocol implementation (OPTIONS, DESCRIBE, SETUP, PLAY, TEARDOWN)
  • SDP generation with H.264 parameters (sprop-parameter-sets)
  • RTP over TCP (interleaved mode)
  • H.264 RTP packetization (RFC 6184 compliant)
  • Multiple clients per camera supported
  • Automatic NAL unit parsing from AVCC format

Configuration

• YAML Configuration System
  • Storage settings (segment duration, path template, format)
  • RTSP server configuration (port)
  • Per-camera configuration (source URL, enabled flag)
  • Credentials embedded in RTSP URLs

Technical Implementation

• Pure Rust: No system dependencies required
• Zero-Copy: Efficient frame handling
• Proper H.264 Handling: AVCC format storage with correct flags

Performance

• Single connection per camera prevents overload
• Broadcast channel supports unlimited subscribers
• Lag handling for slow consumers (skip frames if needed)
• Efficient RTP streaming (90kHz clock, proper timing)

Known Limitations

• H.264 only (no H.265/HEVC support yet)
• Video only (no audio recording)
• TCP interleaved only (no RTP over UDP)
• No DASH output
• No metrics/monitoring UI

Tested Configurations

• ✅ Real RTSP IP camera (1920x1080, 25fps, H.264)
• ✅ Recording with proper MP4 structure
• ✅ Restreaming verified with ffplay and VLC
• ✅ Concurrent recorder + multiple RTSP clients
• ✅ Files playable in VLC/ffplay while recording