Skip to main content

Explanation: Understanding the Labs Project Lifecycle

Introduction​

This document explains the architecture, intent, and governance behind the wwnorton/lab GitLab group — the place where new projects are incubated, forks are matured, and third-party code is evaluated before either being absorbed into a production-owning group or sunset. It covers what the Labs group is for, the states a project moves through, the gates between those states, and the roles that approve each transition. Read the how-to guide first if you need the practical step-by-step procedures.

The Problem: Production-Grade Incubation Is Heavyweight​

Standing up a new application directly in an wwnorton/app/* subgroup carries production expectations from day one: an on-call owner, runbooks, alerting, dependency reviews, security sign-off, and a CI/CD posture that assumes real customers. Those expectations are correct for production services — and exactly wrong for an experiment that may not survive its first month.

Without a dedicated incubation space, teams either skip the rigor (and quietly drag half-baked services toward production) or skip the experiment (because the overhead of "doing it properly" makes the bet uneconomic). Both outcomes are bad: the first erodes reliability of wwnorton/app/*, the second slows down the company's ability to learn.

What Labs Solves​

The Labs group gives experimentation a first-class home with intentionally lighter governance — and an explicit exit. Every project in Labs is on a clock that ends in one of two outcomes:

Owners drive the work. The receiving group lead is the gate at graduation. The Platform team provides shared infrastructure (the labs namespace, the fork pipeline, the security review) without sitting on the approval path.

The Lifecycle State Model​

Every Labs project moves through the same set of states. The transitions explicit events with named approvers.

Time-box reviews while a project is Incubating may extend its target end date — they don't change the state. The transition out of Incubating only happens at graduation or sunset. See Entry Requirements.

A project should be in exactly one state at a time, and the state should be visible in the tracking surface (Jira Epic today; GitLab Issues + Topics going forward — see Tracking Active Projects).

Why an explicit state model? Without it, "old" Labs projects accumulate. They hold namespace quota, keep stale secrets and IAM roles in AWS, confuse new contributors, and quietly start picking up dependencies from other services. The state model exists so that every Labs project has a known next decision date.

Three Project Origins​

Projects arrive in Labs from three distinct places. They share the same lifecycle states but differ in their entry mechanics and — critically — in how they exit at graduation.

1. Fork (most common)​

A fork is a copy of an existing wwnorton/app/* repository created so a team can develop a major change in parallel with production. Common drivers: a large refactor, a commerce-unification cutover, a multi-quarter migration that can't ride on the live trunk.

  • Created via: the Fork Pipeline, which forks both the code and -deploy repositories, builds an image, opens three MRs, and clones AWS secrets into the labs namespace.
  • Graduates by: opening a graduation MR back to the original upstream repository, then sunsetting the Labs fork itself. The Labs fork is the workshop, not the destination.

2. New Project / PoC​

A net-new repository created directly in wwnorton/lab/<initiative>/ — no parent in wwnorton/app/*. Used for proofs of concept, spike implementations, and brand-new services whose owning group is unclear at the outset.

  • Created via: manual repository creation in the appropriate Labs subgroup, followed by manual deploy-repo, FluxCD, and IAM setup.
  • Graduates by: GitLab project transfer into an wwnorton/app/* subgroup chosen by the owner and the receiving group. The Labs path is left behind; history, MRs, and issues move with the project.

3. Imported / Third-Party​

External code being evaluated for absorption: a vendor demo, an OSS project being forked for internal use, a hackathon prototype written outside the org. The project enters Labs with provenance attached (license, attribution, dependency audit) and goes through the same evaluation as a New Project.

  • Created via: GitLab project import (for OSS) or manual upload, plus a provenance attestation in the repo (THIRD_PARTY_PROVENANCE.md or equivalent).
  • Graduates by: GitLab project transfer to a receiving wwnorton/app/* subgroup, same as a New Project. Security review pays particular attention to dependency posture and license compatibility.

The ordering above — Fork, New, Imported — reflects current volume. The Labs group exists primarily as the workshop for forks today; New and Imported are growing usage patterns.

Where Should This Project Live?​

Not every experiment belongs in Labs. The decision tree below covers the common cases.

Two heuristics for the borderline cases:

  • If you can't name the owning group on day one, the project belongs in Labs until that question is answered. Labs exists specifically to defer that decision while still giving you a place to deploy.
  • If you can name the owning group on day one and commit to production go-live, the project belongs in wwnorton/app/<group> directly. Don't use Labs as a "go fast lane" to skip rigor — the production-readiness work has to happen eventually, and doing it later is more expensive than doing it now.

Entry Requirements​

Every project that enters Labs must satisfy three requirements before its state moves from Proposed to Incubating.

1. EM / Tech-lead sign-off​

A named engineering manager or tech lead — distinct from the contributor — must approve the entry. The approver does not become the owner; they are confirming that the work is real, that the contributor has the capacity, and that the Labs slot is justified. Sign-off lives in the tracking surface (Jira Epic comment or GitLab Issue approval).

2. Owner-declared time-box​

The owner declares a target end date for the evaluation at entry. There is no default duration — pick a window honest to the work. A spike to validate a library might be two weeks; a fork carrying a multi-quarter migration might be 180 days.

The time-box is a forcing function, not a hard ceiling. When it expires, the owner must take one of three actions: Graduate, Sunset, or Extend with written justification (and a new end date). Extensions are expected when work is real; the goal is to prevent indefinite drift, not to punish realistic estimates.

3. Written success criteria​

The owner writes down what the evaluation is meant to prove before running it. Examples:

  • "The forked order-service successfully processes 100% of CU integration test traffic with parity on response codes and latency."
  • "The recs-eval PoC produces measurable lift over the current recommender on the offline replay dataset."
  • "The vendor billing engine integrates with our auth, secrets, and observability stack without per-environment customization."

Without written criteria up front, graduation conversations become subjective. With them, the evidence-vs-criteria comparison at graduation time is a paper-trail check.

Placement: subgroup per initiative​

All Labs projects live under a per-initiative subgroup:

wwnorton/lab/
├── commerce-unification/
│   ├── order-service-cu-fork/
│   ├── order-service-cu-fork-deploy/
│   └── catalog-service-cu-fork/
├── recs-platform-eval/
│   └── recs-eval-poc/
└── vendor-billing-eval/
    └── billing-vendor-import/

The subgroup name describes the initiative, not the team. This keeps related projects grouped (the Fork Pipeline already follows this pattern) and makes the eventual transfer or archive easier to scope.

What Labs Is Not​

A short list of things Labs projects must not do. These are non-negotiable because the Labs group's value depends on it staying clearly distinct from production.

No production traffic​

Labs services run in dev, qa, and stg only. They do not receive customer requests. They do not back any production user experience. If users will touch it, it does not belong in Labs.

No production data​

Labs services do not hold PII, payment data, or any other production data. Test fixtures, synthetic data, and replayed traffic on anonymized datasets are fine. Anything that would trigger an incident if dumped is not.

No production-tier backing infrastructure​

Labs projects use the dev/qa/stg-tier RDS instances, S3 buckets, and other shared infrastructure. They do not provision production-tier resources through the self-service Terraform modules — the OPA policies that gate those modules already enforce this for environment names, and a Labs project requesting a production-tier resource is a signal that it has outgrown Labs and should graduate.

No public-facing URLs without Platform review​

Internal *.dev.wwnorton.net / *.qa.wwnorton.net ingresses are fine. A public ingress reachable from the open internet requires Platform team review before provisioning. The review is short — its purpose is to prevent shadow-production deployments that grow real users without ever passing through graduation.

Why "no public URLs" matters. A Labs project that picks up real external users is no longer evaluating — it is operating. Operating services need on-call, alerting, runbooks, and security review, which is exactly what graduation provides. Skipping graduation by quietly serving real users is the failure mode this prohibition exists to prevent.

Tracking Active Projects​

Every Labs project must be visible in a tracking surface so the org can answer "what's in Labs, who owns it, what state is it in, and when is its next decision?" That question is asked at every quarterly portfolio review and during any cross-cutting incident.

Norton is mid-migration on the tracking surface itself: today, Jira Epics are the source of truth; the migration target is GitLab Issues + Topics. Both must be maintained in parallel during the transition. The how-to guide documents both procedures explicitly.

Tracking model — today: Jira Epics​

Each Labs project has one Jira Epic in the team's project. The Epic carries the project metadata the rest of the org needs:

Epic fieldWhat it holds
Title[Labs] <project-name> — the [Labs] prefix makes filtering across projects trivial
OwnerSingle named owner (assignee). Co-owners go in description.
EM/Lead approverNamed in description. The sign-off comment establishes the entry approval.
OriginFork / New / Imported. Determines graduation path.
Entry dateDate the Epic moved to "In Progress".
Target end dateThe declared time-box.
Success criteriaWritten into the Epic description before the state changes to "In Progress".
Lifecycle stateCustom field or label: proposed, incubating, graduating, sunsetting, archived.
GitLab project URLDirect link(s) to the Labs project(s) — including the -deploy repo for forks.

Quarterly review queries the project board for Epics labeled incubating whose target end date has passed.

Tracking model — going forward: GitLab Issues + Topics​

When the Jira → GitLab migration completes, the tracking surface moves into GitLab itself. The state of a Labs project becomes visible from the project page and the group landing page rather than from a separate tool.

GitLab Topics on the project (set at creation, updated on state transitions):

TopicWhen to apply
labs-proposedEntry MR drafted; awaiting EM/Lead sign-off
labs-incubatingActive evaluation
labs-graduatingGraduation requested; receiving group lead reviewing
labs-sunsettingTeardown in progress
labs-archivedRepo archived; no further work
origin-forkSet once at creation; never changes
origin-newSet once at creation; never changes
origin-importedSet once at creation; never changes

GitLab Issues in a dedicated wwnorton/lab/labs-registry project capture the same fields the Jira Epic does today (owner, EM/lead approver, entry/end dates, success criteria, lifecycle state). One issue per Labs project. State transitions are recorded as comments and label changes.

The advantage of the GitLab model is locality: all the metadata for a Labs project lives in the same tool as the code and the deploy artifacts. The disadvantage is that GitLab issue search across groups is less mature than Jira's reporting — which is why the migration is incremental rather than a flag-day.

During the transition, both must agree. If you change state in Jira, update the GitLab topics on the same day. If you graduate a project via GitLab, close the Jira Epic on the same day. The how-to guide treats both as required steps in the same procedure.

Graduation: Two Paths​

Graduation is where Labs returns its investment. The mechanics differ sharply based on origin.

Path A — Fork Origin

Path B — New / Imported Origin

Why Path A (fork) ends with a sunset​

A Labs fork is a workshop, not a destination. The changes developed there exist to be applied to the upstream repo. Once the graduation MR is merged into upstream, the fork has no future use — keeping it running would mean two deployments of the same logical service competing for namespace, secrets, and ingress. The fork is sunset using the same procedure as any other Labs cleanup.

Why Path B (new / imported) ends with a transfer​

A New or Imported project is its own thing — there is no upstream to merge back to. Graduation moves the project itself into its production home via GitLab's native project transfer, which preserves history, merge requests, issues, and CI configuration. The deploy artifacts (FluxCD, AWS secrets, IAM) are migrated alongside the code.

Graduation prerequisites (both paths)​

Before requesting graduation approval, the owner must complete the following. These are not "nice-to-haves" — graduation is blocked without them.

  1. Security review. Platform/security signs off on the secrets layout, IAM configuration, and dependency scan results. The review output is attached to the tracking record.
  2. CI tests passing + coverage threshold met. Real test suite (not smoke), at or above the coverage bar set by the receiving group. Both the value and the source of the bar are recorded in the tracking record.
  3. Evidence against the entry-time success criteria. The criteria written at entry are compared point-by-point against the evidence the evaluation produced.

The receiving group lead is the sole approver of graduation. The Platform team is informed (and is the security reviewer above) but does not gate the approval itself.

No production-readiness items beyond those three are mandatory. On-call, Grafana dashboards, runbooks, and alerts are expected of every production service, but Labs governance does not enforce them — that is the receiving group's responsibility once they accept the project. The graduation handshake is the receiving group lead saying "we will own this, including its operability gaps."

Sunset: Two Triggers​

A project enters the Sunsetting state from one of two sources:

  1. Failed evaluation. The success criteria were not met, the experiment is dead, or the time-box expired without a justified extension. The owner declares the outcome in the tracking record.
  2. Post-graduation fork cleanup. The graduation MR for a Fork-origin project merged into upstream. The Labs fork is now redundant and must be removed.

Both triggers run the same teardown sequence. The how-to guide walks through it step by step; the explanation below covers the why of the ordering.

Sunset ordering: why FluxCD first, archive last​

Tearing down deploy artifacts before archiving the source repo prevents orphan resources from being stranded behind a read-only project. Tearing them down in the order FluxCD → secrets → IAM → archive matches the dependency direction:

  • FluxCD first removes the running pods, which stops anything from trying to read the secrets or assume the IAM role.
  • Secrets next removes the values from AWS Secrets Manager — scheduled with a recovery window, not deleted immediately, so accidental sunset can be reversed.
  • IAM config next removes the application entry from config.{dev,qa,stg}.json so the next terraform apply destroys the service account and IAM role.
  • Archive last because the repos are still useful for reference during the cleanup itself. Archiving makes them read-only but preserves history.

Data resources created via the self-service Terraform modules (S3 buckets, RDS instances) are removed by the owner through the same workflow that created them — an MR to the relevant terraform tfvars file. This keeps the audit trail intact and avoids Platform-team bottlenecks.

Archive, don't delete. GitLab archive makes a project read-only but keeps everything — history, MRs, issues, CI runs. Hard delete loses all of it. For a Labs project that has been live for weeks or months, the archived repo is often the cheapest place to find the actual code history of "what did we try and why didn't it work" the next time someone proposes a similar evaluation.

Roles & Approvals​

The Labs lifecycle is intentionally short on approvers. Four roles cover the entire flow.

Project Owner​

The single named human responsible for the Labs project. The owner:

  • Drafts the entry record (Jira Epic and/or GitLab Issue + Topics).
  • Declares the time-box and writes the success criteria.
  • Executes the day-to-day work, including the creation/fork procedures, the time-box reviews, and the teardown.
  • Requests graduation or sunset when the time comes.

Co-owners are allowed and encouraged, but one human's name is on the record.

Engineering Manager / Tech Lead (entry approver)​

Approves the move from Proposed to Incubating. The EM/Lead does not become the owner; they are confirming the work is real, the contributor has capacity, and the Labs slot is justified. Sign-off lives as a comment on the tracking record.

Receiving Group Lead (graduation approver)​

The tech lead of the wwnorton/app/* subgroup that will own the project after graduation. Sole approver of the Graduating → Migrated transition. By approving, the receiving group lead commits their team to ownership — including any operability gaps the project carries.

For Fork-origin projects, the receiving group lead is typically the tech lead of the same group that owns the upstream repo. For New / Imported projects, the receiving group is identified by the owner and the receiving group lead together during the graduation conversation.

Platform Team​

Provides the shared infrastructure that makes Labs work (the labs namespace, the Fork Pipeline, the self-service terraform modules, the secrets pipeline) and performs the security review required for graduation. The Platform team is informed about graduation and sunset events but is not a gate on the graduation approval itself.

The Platform team is a gate on two specific Labs operations:

  • Provisioning a public-facing URL for a Labs service (Platform must review before the ingress is created).
  • Requests for production-tier backing infrastructure from a Labs project (Platform will decline and direct the owner to graduate first).

FAQ​

Why is there no default time-box?​

Because the right duration depends on the work. A library evaluation might genuinely be done in two weeks; a fork carrying a six-month migration is honest at 180 days. Forcing a single default would either rush real work into Sunset or rubber-stamp extensions on lazy estimates. The forcing function is the declaration, not the number.

What stops a Labs project from running forever?​

The combination of the declared end date, the requirement to take a documented action when the date arrives (Graduate / Sunset / Extend-with-justification), and the quarterly portfolio review that scans for expired records. Extensions are allowed and expected, but each extension is a visible decision — not a silent default.

Can a project graduate without a Platform-team security review?​

No. The security review is one of three hard prerequisites (the others being CI tests/coverage and evidence against success criteria). The Platform team is not the graduation approver — that is the receiving group lead — but the security review is a non-negotiable checklist item the owner must clear before graduation can be approved.

Can the receiving group at graduation time be different from the team that worked on the Labs project?​

Yes, frequently. A Labs project may be developed by a small group that does not have a long-term home for it; graduation is the moment the receiving group is identified. The owner and the receiving group lead negotiate the handoff together, and the receiving group lead is the approver — they cannot be conscripted.

What happens if the receiving group declines graduation?​

The project goes from Graduating back to Incubating with a new time-box and addressed concerns, or it goes to Sunsetting if the concerns can't be addressed. A decline is not a failure — it is the receiving group exercising the ownership decision the gate exists to protect.

Does an extension restart the time-box clock from zero?​

No. An extension sets a new end date with written justification, but the entry date and the original criteria do not change. The tracking record carries both — the original time-box and every extension — so the full duration is visible at portfolio review.

Is the Labs fork sunset the same procedure as a failed-eval sunset?​

Yes. Both run the same FluxCD → secrets → IAM → archive sequence. The only difference is which trigger appears in the tracking record. The how-to guide documents one procedure used in two situations.

What about Labs projects that produce shared libraries or tooling instead of services?​

A Labs project that produces a library (not a deployed service) skips the deploy-artifact migration during graduation — there's nothing to FluxCD-migrate. Graduation for library-only projects is just the GitLab project transfer (Path B), with the security review focusing on the package's dependency posture. The lifecycle states, gates, and approvers are unchanged.

Where can I see what's currently in Labs?​

Until the GitLab Issues + Topics tracking model is fully rolled out, the source of truth is the Jira board filtered to Epics with the [Labs] prefix. After rollout, the wwnorton/lab/labs-registry project will hold the issue-per-project records, and a GitLab Topics search across wwnorton/lab will surface the projects directly.