Deployment and operations

Cahier §13 asks for three environments (développement, préproduction / recette client, production, §13.1), a deployment that covers « serveur, base de données, stockage et sauvegardes », « domaines, certificats et services externes » and the « import initial des produits, revendeurs et comptes utilisateurs » (§13.2), training (§13.3) and a maintenance model with « délais de prise en charge et niveaux de priorité » (§13.4). The requirements are OPS-01 to OPS-06; the work is planned in OPS-01, OPS-02, SRV-28, MOB-14 and the launch tasks LCH-01 to LCH-05. Dates are on the timeline.

Timing within the three-month frame. V1 — design, development and deployment, the reseller web included — fits in three months from the cadrage. Production goes live at the start of January (milestone Production go-live), the two-week pilot starts right after (Pilot starts), and generalisation and V2 come after the three months. The consequences for operations:

3-month frame: design → development → deployment — deadline 17 Jan 2027.

1. Acceptance falls around the year end. The functional campaign (QA-03), the end-to-end tests (QA-01) and the client acceptance (QA-02) run back to back over the last working days of December and the first of January. HOCO's testers, the PV signatory and the import data owners are booked at the cadrage, with named back-ups, so that year-end absences do not stop the recette.
2. Production is prepared before the recette, not after it. The acceptance report is signed one working day before go-live, so everything that does not depend on the accepted build — infrastructure, domains, backups, monitoring, the SMS sender ID, the Play Console listing, the privacy policy, the import rehearsal — is done during the build phase (go-live checklist).
3. No slack on the critical lane. Anis's lane fills the three months; an operational task that slips (sender-ID approval, store review, data delivery) is escalated the same week and handled with the levers of the timeline and D-38.

EnvironmentsCopy Anchor Link

hoco.dz is a placeholder until D-12; the admin, app and cdn names are proposals.

Assumption — the pre-production apps are the staging flavor with API_BASE_URL set at build time (--dart-define), so no fourth flavor is needed; Imad confirms in the mobile guide.

Data policy per environment

Delivery pipelineCopy Anchor Link

Server, back-office and reseller webCopy Anchor Link

Rules:

1. One image, two processes. api (HTTP) and worker (BullMQ workers and the scheduler) run from the same image with different commands; api runs two replicas in production, worker one.
2. Migrations are expand-then-contract. A release only adds columns, tables and indexes, or relaxes constraints; removals ship one release later, so the previous image can always run against the new schema — that is what makes a rollback safe.
3. Rollback = redeploy the previous image tag in Dokploy (runbook); a database rollback is a restore, never a down-migration.
4. Feature flags (settings) hide unfinished V1 features rather than long-lived branches.

Mobile appsCopy Anchor Link

Imad's repository hoco-mobile, GitHub Actions (MOB-01, MOB-14):

versionName follows semantic versioning; versionCode is the CI build number. The minimum and latest versions per app and platform are set on BO-66 (AppVersionPolicy); raising the minimum forces an update (426 UPGRADE_REQUIRED). A new server release that mobile depends on is deployed before the app release that uses it.

Infrastructure sizingCopy Anchor Link

Proposal for the volumes of the sizing table (Assumption; provider and data location are D-15).

Headroom: the production VPS runs the generalisation volume at about a third of its capacity; the next steps are a larger VPS, then a separate database server, then more API replicas (NFR-03).

Domains and TLSCopy Anchor Link

Invitation links work from the first production day. The reseller web is part of V1 (WEB-01 to WEB-05) and goes live with production (LCH-01); it serves the /i/{code} landing (open the app, else the Play Store with the referrer, else the web sign-up) and /.well-known/assetlinks.json, so the invitations delegates share during the pilot resolve correctly. The fingerprints in assetlinks.json are those of the production Play app-signing key, known once the app exists in the Play Console (MOB-14).

HSTS is enabled per host first, then with includeSubDomains once every subdomain serves HTTPS (SEC-03).

SecretsCopy Anchor Link

The env package validates every variable at boot; a missing or malformed secret stops the process with a clear message. Secrets never appear in the repositories (gitleaks) or in logs (SEC-0.7).

BackupsCopy Anchor Link

A verification job at 04:00 checks that last night's dump exists, is larger than 50 % of the previous one and can be listed by pg_restore; otherwise it alerts. A full restore is tested every month (and in SRV-28 before go-live).

Restore runbookCopy Anchor Link

1. Declare the incident (P1), name the person in charge, inform HOCO's referent.
2. Stop writes: switch maintenance mode on (BO-66 or the settings switch; the API answers 503 MAINTENANCE); stop the worker. The delegate app keeps working offline and queues its actions.
3. Choose the recovery point: the latest consistent point (point-in-time recovery) or the last good dump; write it down.
4. Provision a fresh database (new instance or container); never restore over the damaged one, keep it for analysis.
5. Fetch and decrypt the base backup and WAL (or the dump) from the backup target with the offline key.
6. Restore: point-in-time recovery to the chosen time, or pg_restore --no-owner --clean --if-exists into the fresh database.
7. Check integrity: prisma migrate status clean; row counts of key tables against the last metrics; the last CMD-, FAC-, PAY- and RET- numbers against NumberSequence; outbox rows not yet dispatched listed.
8. Switch DATABASE_URL of api and worker to the restored database; start the worker (it re-dispatches pending outbox events).
9. Smoke test: sign-in, catalog, the orders-to-confirm queue, a test order on a test reseller, a delegate sync.
10. Reopen: maintenance off; delegates' queued actions replay; announce the end of the incident.
11. Report within two working days: cause, data lost (between the recovery point and the incident), duration against the 4-hour objective, actions.

Restore drillCopy Anchor Link

The drill proves step 4 to 9 of the runbook on a fresh database without touching the live one. The smoke check is docker/restore-smoke.sql (row counts of the key tables, pending outbox rows, stock totals and the NumberSequence values): run it on the source and on the restored database and compare.

# 1. dump (inside the PostgreSQL container; on the VPS: fetch and decrypt last night's dump instead)
docker exec postgres pg_dump -U postgres -Fc -f /tmp/hoco-drill.dump hoco
# 2. restore into a fresh database
docker exec postgres createdb -U postgres hoco_restore
docker exec postgres pg_restore -U postgres -d hoco_restore --no-owner --exit-on-error /tmp/hoco-drill.dump
# 3. smoke: identical counts and sequences, then the API against it
docker cp docker/restore-smoke.sql postgres:/tmp/restore-smoke.sql
docker exec postgres psql -U postgres -d hoco -At -f /tmp/restore-smoke.sql > source.txt
docker exec postgres psql -U postgres -d hoco_restore -At -f /tmp/restore-smoke.sql > restored.txt
diff source.txt restored.txt
DATABASE_URL=…/hoco_restore  # an api process on it: GET /ready, sign-in, catalog, admin.orders.list
# 4. clean up
docker exec postgres dropdb -U postgres hoco_restore

MonitoringCopy Anchor Link

Alert thresholdsCopy Anchor Link

Proposed thresholds; alerts go by email and to the team's chat channel; during the pilot P1 alerts also call the on-call phone.

Initial data importCopy Anchor Link

Cahier §13.2 « Import initial des produits, revendeurs et comptes utilisateurs »; ADM-06, CAT-06; built in SRV-27 (import jobs with dry-run report, idempotent apply and error rows), run in LCH-01; sources and format are D-31.

Cleansing rules applied by the dry-run and reported row by row: phones normalised to E.164, duplicates by normalised phone listed for HOCO to merge before import (CRM-03); communes matched to the official reference with suggestions for near spellings (Bir el Djir → Bir El Djir); brands limited to HOCO, ACEFAST and BOROFONE; EAN-13 checksums verified; prices positive and in the order super-gros ≤ demi-gros ≤ détail (warning otherwise); VAT 19 % when empty; missing GPS accepted (captured at the first visit); a reseller without a customer type is rejected.

Steps

1. Templates and samples — HOCO receives the templates and returns 20 sample rows per data set as early as possible, so the templates are tuned before SRV-27 ends.
2. Full extract — HOCO's data owners fill the templates; Anis runs the first dry-run on pre-production and shares the report.
3. Cleansing rounds — HOCO corrects; dry-run again until zero errors (warnings reviewed).
4. Rehearsal — full import on pre-production; HOCO checks 20 random products (prices of the three types, attributes, images) and 20 random resellers (customer type, delegate, wilaya) and signs the check.
5. Freeze — from the agreed freeze moment, changes are made by hand in the new back-office; stock and opening balances are taken at the freeze.
6. Production import during the go-live window in the order of the table; each apply is idempotent and can be re-run.
7. Reconciliation — counts per data set, total stock per warehouse and total credit limits compared with HOCO's figures; signed by HOCO.

Cut-over. During the pilot, non-pilot resellers keep ordering by phone and WhatsApp as today; orders of pilot resellers received by phone are entered by staff on BO-40 so every pilot order is in the platform and goes through the « Commandes à confirmer » queue. From go-live the platform is the stock reference for every order it handles (D-34).

TrainingCopy Anchor Link

Cahier §13.3: administrators and managers, delegates on a full tour scenario, a quick guide for resellers, administration documentation and support procedures (OPS-03). Materials by Akram with Anis (DQA-06); sessions in LCH-02 on pre-production with the recette data, repeated for each generalisation wave (LCH-05).

Train-the-trainer: HOCO names one champion per function who trains later arrivals with the same materials. The administration documentation and support procedures (L1 playbook) are handed over at generalisation.

Go-live checklistCopy Anchor Link

Keyed to the milestones of the plan, because acceptance ends one working day before go-live. Owner in brackets.

During the build — before Staging integration complete

1. Production infrastructure provisioned and hardened (SEC-0.10), DNS and TLS live, HSTS per host — done with OPS-02 (Anis).
2. Backups running and one full restore tested in SRV-28 (Anis).
3. Monitoring and alerts live; a test alert fired and received (Anis).
4. SMS sender ID approved; a real OTP received on a Mobilis, a Djezzy and an Ooredoo number — the registration is started at the cadrage because approval takes time (HOCO, Anis).
5. Firebase hoco-prod, production Maps keys restricted, email SPF/DKIM/DMARC passing (Anis, Imad).
6. Play Console: apps created, closed track with the pilot users' emails, data safety form, foreground-service declaration, privacy policy URL, reviewer demo accounts — MOB-14, leaving the store review time before go-live (Imad).
7. Privacy policy and account-deletion page published (HOCO, Anis).
8. HOCO's testers, PV signatory and data owners booked for the year-end recette, with back-ups (HOCO, Anis).

During the recette — before Acceptance report (PV de recette) signed

1. Business settings validated by HOCO on pre-production: minimum order, payment methods, credit policy, reservation status, GPS tolerance, assisted-order delay, return window, quiet hours (HOCO, Anis).
2. Full import rehearsed on pre-production with zero errors; reference lists, attribute definitions and support regions with their WhatsApp numbers ready (HOCO, Anis).
3. Staff accounts listed; training sessions and the pilot delegates' half-day scheduled; champions named (Anis, Akram).
4. Pilot delegates' phones checked (Google Play services, battery optimisation exclusion); tours of the first pilot week drafted (Akram, HOCO).
5. Go/no-go meeting at the end of the recette: open defects against the exit criteria, risks, rollback plan (all).

Go-live day — Production go-live

1. Freeze; production import in order; reconciliation signed (Anis, HOCO).
2. Production release deployed (API, back-office, reseller web with the invitation landing and assetlinks.json); smoke test on production with test accounts (Anis).
3. Apps promoted to the closed track; pilot users invited (Imad).
4. Support channels staffed (regional WhatsApp numbers, phone), escalation contacts shared (HOCO).

Next working day — Pilot starts

1. Training of administrators, managers and pilot delegates (LCH-02) (Anis, Akram).
2. Morning check: errors, queues, orders to confirm, sync errors, SMS delivery (Anis, Imad); first daily pilot triage (LCH-03, LCH-03B).

Pilot and generalisationCopy Anchor Link

The pilot runs two weeks right after go-live, with a limited group of delegates and resellers using the two apps and the reseller web (OPS-06; group and length in D-32). It ends before Ramadan, so field activity is representative. Daily triage at the end of the field day; hotfixes on the closed track. Proposed go/no-go criteria for the Pilot go / no-go milestone:

Generalisation comes after the three months and opens zone by zone (LCH-05): train the zone's delegates, import or validate its resellers, promote the apps to the public Play track and announce the reseller web to the zone; one wave per week as long as the support load stays within the targets below. The iOS build, if HOCO decides it (D-10), ships in this phase (MOB-16).

Maintenance and supportCopy Anchor Link

Cahier §13.4: « Période de garantie corrective à préciser », « Maintenance corrective, préventive et évolutive selon contrat », « Support, délais de prise en charge et niveaux de priorité », « Suivi des performances, sauvegardes et mises à jour de sécurité » (OPS-04). Everything below is proposed and becomes binding through the contract.

Warranty. Two months of corrective warranty from production go-live (covering the pilot and generalisation): defects in the delivered and accepted scope are fixed at no cost.

Support levels and channels

Priorities and targets (business hours Sunday–Thursday 08:30–17:30 Africa/Algiers; during the pilot, P1 is covered every day 08:00–20:00)

Recette severities map to these priorities: Blocker ≈ P1, Critical ≈ P2, Major ≈ P3, Minor ≈ P4 (acceptance procedure).

Releases after go-live: hotfixes any day for P1 and P2; a minor release every two weeks during pilot and generalisation, then monthly. Reporting: a monthly report to HOCO with tickets by priority and targets met, availability, incidents, backups and restore tests, security updates and the next month's plan.

RunbooksCopy Anchor Link

Each runbook lives in the operations repository folder and is rehearsed on pre-production before go-live.