Intelletto.ai Resume Scoring

Production System Reference

Version Block

• document_version: 1.0.0
• document_effective_date: 2026-04-04
• status: Production System Reference
• pipeline_version: v3 (Cloud Run, revision 276+)
• scoring_engine_version: 1.0.0
• scoring_config_schema_version: scoring_config.schema.v1
• scorecard_schema_version: scorecard.schema.v1
• model_id (Gemini): gemini-2.5-flash (bounded inference only, with response_schema enforcement)
• taxonomy_snapshot_id: taxsnap_<ulid> (immutable snapshot reference used for scoring run)
• test_suite: 747 tests passing (v3: 101, v2 reference: 327, integration: 319)

Ownership and Governance

• Executive Owner: Scott Darrow
• Product Owner: Scott Darrow
• Engineering Owner: Intelletto Platform Engineering
• Change control rule: All changes to scoring config schema, scorecard schema, or bucket rubrics require version bump, backward compatibility statement, and regression test updates (golden packs).

Document Section Index

• Section 1: System Architecture (v3)
• Section 2: 8 Scoring Buckets (Formulas and Implementation)
• Section 3: Scoring Gates
• Section 4: JD Skill Requirements Loading
• Section 5: Modifiers
• Section 6: Scoring Config Lifecycle
• Section 7: Database Schema
• Section 8: API Endpoints (v3)
• Section 9: Enhancement Endpoints
• Section 10: Scoring Runtime Workflow
• Section 11: Environment and Deployment
• Section 12: Known Gaps and P0 Status
• Section 13: Non-Negotiable Invariants

Section 1: System Architecture (v3)

1.1 Runtime Environment

• Runtime: Google Cloud Run (production: api.intelletto.ai)
• Database: Google Cloud SQL PostgreSQL (intelletto-ai, asia-southeast1), schema: intelletto, ~169 tables
• Storage: Google Cloud Storage for resume ingestion (RESUME-POOL/) and archival
• OCR: Google Document AI for layout parsing
• LLM: Google Gemini 2.5 Flash for structured extraction, enrichment, and soft signal inference (bounded inference only, always with response_schema enforcement)
• Entry point: v3/main.py
• App factory: v3/src/api/app.py
• Local dev server: http://localhost:8080

1.2 v3 Scoring Domain Modules

The scoring engine is organized as a domain module within the v3 architecture:

1.3 Shared Scoring Infrastructure

1.4 Upstream Pipeline Stages (12-Stage)

Scoring starts only after the resume parsing pipeline completes stages 01–09 and creates a scoring_input_snapshot. The pipeline is orchestrated by Netflix Conductor (workflow intelletto_pipeline_mode_c), with workers on Cloud Run (intelletto-worker). Conductor manages stage ordering, retries, rate limits, decision branches, and HUMAN review gates. Stage 11 (Scorecard) is a Conductor task executed by the worker's scorecard.py handler.

1.5 Component Boundaries

• JD Orchestrator owns: JD authoring, versioning, requirements, must-haves, proficiency/recency targets, AI JD generation
• Resume Parsing Pipeline owns: extraction, evidence mapping, normalization, canonical skill candidates, scoring input snapshot creation
• Scoring Engine owns: gate evaluation, bucket scoring, modifier computation, scorecard persistence, explanations and evidence linkage
• Gemini owns nothing persistently: produces bounded artifacts stored in Postgres and referenced by IDs. Always invoked with response_schema for structured output enforcement

Section 2: 8 Scoring Buckets (Formulas and Implementation)

Each bucket produces a score in the range 0–100. Buckets are combined via configurable weights (must sum to 1.0 for enabled buckets) into base_fit. Bucket dispatch uses ScorerRegistry in src/scoring/runtime.py with exact match on bucket_id from the published scoring config.

Default bucket weights (production):

2.1 Hard Skills / Tools (default 60%)

Implementation: _score_bucket_hard_skills

Formula:

score = required_match_rate * 75% + nice_to_have_rate * 25% + quality_bonus (up to +8)

Match tiers:

• Exact match: 1.0
• Alias match (via skill_alias table): ≥0.85
• Fuzzy match: ≥0.65

Inputs:

• Candidate normalized skills (from normalization_result) plus certification-expanded skills (from certification_skill_map)
• JD required and nice-to-have skills (loaded via 3-level fallback; see Section 4)

2.2 Soft / Behavioral (default 20%)

Implementation: _score_bucket_soft_behavioral

Formula:

12 behavioral signals scored from evidence spans
leadership and problem_solving weighted 1.2x (elevated importance)
All other signals weighted 1.0x

Uses soft signal inference from Gemini (bounded, schema-validated) or deterministic pattern matching from parsed resume text. Signals include: leadership, problem_solving, collaboration, ownership, adaptability, communication, attention_to_detail, initiative, mentoring, strategic_thinking, stakeholder_management, conflict_resolution.

2.3 Domain / Process (default 16%)

Implementation: _score_bucket_domain_process

Formula:

score = title_relevance * 50% + skill_domain_overlap * 30% + industry_continuity * 20%

• title_relevance: semantic similarity between candidate's current/recent titles and JD title
• skill_domain_overlap: ratio of JD domain-specific skills present in candidate profile
• industry_continuity: consistency of industry across candidate's work history relative to JD industry

2.4 Languages / Communications (default 2%)

Implementation: _score_bucket_languages_comms

Formula:

score = english_score * 60% + additional_languages * 40%

• english_score: CEFR-level assessment from resume text quality (C2=100, C1=85, B2=70, B1=50, A2=30, A1=15)
• additional_languages: bonus for languages matching JD requirements

2.5 Scope / Complexity (default 1%)

Implementation: _score_bucket_scope_complexity

Formula:

score = seniority_score * 60% + scope_signals * 40%

Seniority pattern matching:

scope_signals: team size mentions, budget mentions, multi-location/global indicators, cross-functional evidence.

2.6 Education / Certifications (default 1%)

Implementation: src/scoring/education_scorer.py

Tenure decay model:

EducationScore = (DegreeScore x DecayFactor x DegreeWeight) + (SkillsMatchScore x SkillsWeight)

DecayFactor = e^(-0.18 x max(0, YearsExperience - 3))

Combined bucket = education_scorer * 55% + cert_matching * 35% + cert_expanded_skill_bonus * 10%

Seniority-aware degree vs skills weighting:

Additional factors:

• Field relevance: 1.0 (exact match to JD field) down to 0.5 (no match)
• Skill recency: current = 1.0, 2–4 years = 0.75, 5+ years = 0.50
• Certification bonus: 0.04 per matched cert, capped at 0.12; expiry-aware
• Vendor cert matching: experience-weighted, specializations 1.5x multiplier

2.7 Tenure / Recency

Implementation: _score_bucket_tenure_recency

Formula:

score = years_score * 40% + recency_score * 35% + stability_score * 25%

• years_score: total relevant experience years vs JD requirements
• recency_score: how recently the candidate used JD-relevant skills
• stability_score: average tenure per role, penalizing job-hopping patterns

2.8 Compliance / Scheduling

Implementation: _score_bucket_compliance_scheduling

Formula:

score = completeness * 50% + norm_confidence * 30% + gate_health * 20%

• completeness: resume section completeness (work history, education, skills all present)
• norm_confidence: average normalization confidence across matched skills
• gate_health: proportion of upstream validation gates that passed

2.9 Scoring Math (Deterministic)

base_fit      = SUM(bucket_weight_i * bucket_score_i)  for enabled buckets where SUM(weights) = 1.0
modifier_pts  = clamp(SUM(modifier_points_j), -budget, +budget)
score_total   = clamp(base_fit + modifier_pts, 0, 100)

If DISQUALIFIED: score_total = null, base_fit = null, modifier_points = null

Score band mapping (src/scoring/helpers.py):

Section 3: Scoring Gates

Gates are fail-fast rules evaluated before bucket scoring. If any gate with severity DISQUALIFY fails, scoring stops and the scorecard status is set to DISQUALIFIED with score_total = null.

3.1 Gate Types (DSL)

3.2 Gate Severities

3.3 Upstream Pipeline Validation Gates

Five gates control pipeline progression before scoring. Each writes a passed boolean to pipeline_gate_result:

Section 4: JD Skill Requirements Loading

_load_jd_skill_requirements() implements a 3-level fallback to ensure JD skills are available for scoring regardless of how they were authored:

1. Level 1: jd_skill_requirement table (preferred)
   • Structured rows with skill_id, importance (required / nice_to_have), proficiency targets, recency constraints
   • Written by the JD Orchestrator when requirements are explicitly configured
2. Level 2: job_description_version.extracted_features_json (legacy)
   • JSONB column containing skills extracted from JD text by earlier pipeline versions
   • Used when jd_skill_requirement rows do not exist for the JD version
3. Level 3: job_description_version.model_json (JD Orchestrator)
   • Full JD model from the JD Orchestrator UI, containing skills within the structured JD payload
   • Ensures skills saved via AI JD Generator are available for scoring even when jd_skill_requirement rows have not been synced

This three-level fallback guarantees that no matter how a JD was created (manually structured, legacy extraction, or AI-generated), the scoring engine has access to skill requirements.

Section 5: Modifiers

Modifiers are post-bucket point adjustments, bounded and configurable. They are applied after base_fit computation and clamped by per-modifier min_points / max_points and a global modifiers_budget_points.

5.1 Data Fusion Confidence

points = ((avg_norm_confidence - 0.5) / 0.5) * max_points

Where:
  avg_norm_confidence = average confidence score across all normalization results
  max_points = configured maximum for this modifier (e.g., 3)

Result range: -max_points to +max_points
  Confidence 0.5 = 0 points (neutral)
  Confidence 1.0 = +max_points (full bonus)
  Confidence 0.0 = -max_points (full penalty)

5.2 Evidence Coverage JD Duties

points = ((jd_coverage_ratio - 0.5) / 0.5) * max_points

Where:
  jd_coverage_ratio = proportion of JD duties evidenced in candidate resume
  max_points = configured maximum for this modifier (e.g., 4)

Result range: -max_points to +max_points
  Coverage 50% = 0 points (neutral)
  Coverage 100% = +max_points (full bonus)
  Coverage 0% = -max_points (full penalty)

5.3 Modifier Persistence

Each modifier result is persisted to intelletto.modifier_result with: modifier_id, modifier_name, points_awarded, min_points, max_points, basis_json (computation inputs), and evidence_json.

Section 6: Scoring Config Lifecycle

6.1 Config States

DRAFT  -->  PUBLISHED  -->  DEPRECATED
  |                           |
  |  (mutable, editable)      |  (immutable, frozen)
  |                           |
  +--- New DRAFT forked ----->+  (append-only versioning)

6.2 Governance Rules

• Append-only versioning: Edits to a PUBLISHED config are rejected. A new DRAFT must be created (optionally cloned from the published version).
• Publish validation gate: Before a config transitions to PUBLISHED, the system validates:
  • Enabled bucket weights sum to 1.0 (within tolerance ±0.0001)
  • Gate DSL types are supported and schema-valid
  • Modifier clamp bounds are consistent
  • Prohibited features are not referenced
• Rollback: Reactivates a prior PUBLISHED version for new scoring runs without altering history. All existing scorecards remain associated with the config used at scoring time.
• Runtime enforcement: Scoring runtime rejects non-PUBLISHED configs with CONFIG_NOT_PUBLISHED.

6.3 Config Payload Shape (JSONB)

{
  "gates": [
    {
      "gateId": "gate_required_skills",
      "name": "Required Hard Skills (N-of-M)",
      "severity": "DISQUALIFY",
      "rule": {
        "type": "N_OF_M_SKILLS",
        "minRequired": 4,
        "skillIds": ["skill_sql", "skill_excel", "skill_o2c", "skill_sap_fi", "skill_oracle_ebs"]
      }
    }
  ],
  "buckets": [
    {"bucketId": "hard_skills_tools", "weight": 0.60, "enabled": true},
    {"bucketId": "soft_behavioral", "weight": 0.20, "enabled": true},
    {"bucketId": "domain_process", "weight": 0.16, "enabled": true},
    {"bucketId": "languages_comms", "weight": 0.02, "enabled": true},
    {"bucketId": "scope_complexity", "weight": 0.01, "enabled": true},
    {"bucketId": "education_certs", "weight": 0.01, "enabled": true}
  ],
  "modifiers": [
    {"modifierId": "data_fusion_confidence", "minPoints": -3, "maxPoints": 3},
    {"modifierId": "evidence_coverage_jd_duties", "minPoints": -4, "maxPoints": 4}
  ],
  "modifiersBudgetPoints": 12
}

Section 7: Database Schema

All tables reside in the intelletto schema on Cloud SQL PostgreSQL (intelletto-ai, asia-southeast1).

7.1 scorecard_version (primary scoring output)

7.2 bucket_score (per-bucket breakdown)

7.3 modifier_result (per-modifier breakdown)

7.4 scoring_gate_result (per-gate evaluation)

7.5 scoring_config_version (config governance)

7.6 scoring_run_metering (operational telemetry)

7.7 Related Pipeline Tables

The scoring engine consumes data from these upstream tables (read-only during scoring):

• scoring_input_snapshot — immutable scoring handoff payload with normalized skills, work history, education, certifications, languages, gate status
• parsed_resume — structured extraction output with sections_json, extraction_json, evidence_spans_json
• normalization_result — canonical skill mappings (raw_term → canonical_skill_id with match_type and confidence)
• pipeline_gate_result — gate pass/fail outcomes per pipeline run (Gates A through E)
• enrichment_payload — data fusion results
• candidate_profile — canonical candidate record

Section 8: API Endpoints (v3)

All endpoints are served from the v3 application on Cloud Run. API namespace: /api/v3/.

8.1 Core Scoring Endpoints

8.2 Scoring Analysis Endpoints

8.3 Internal Pipeline Endpoints (v1 bridge)

The pipeline internal endpoints are served via the v3 v1-native bridge (v3/src/api/middleware/v1_proxy.py). Key scoring-related internal routes:

8.4 Example API Payloads

Score Request (via pipeline)

POST /api/v1/scoring/scorecards
{
  "scoringConfigVersionId": "scv_2026_04_01_0001",
  "jdVersionId": "jdv_01J4QZ0B1J8A1D7S3H3F2C9M11",
  "parsedResumeId": "pr_01J4QZ2Z1S5JZ9RZB9E1A2B8C3",
  "runMode": "PERSIST",
  "requestId": "req_01J5A1B2C3D4E5F6G7H8"
}

Response (SCORED)

{
  "scorecardId": "sc_01J5S2K7A2T9K3H8Q6D1E4F7G9",
  "status": "SCORED",
  "scoreTotal": 82.6,
  "baseFit": 79.4,
  "modifierPoints": 3.2,
  "scoringConfigVersionId": "scv_2026_04_01_0001",
  "jdVersionId": "jdv_01J4QZ0B1J8A1D7S3H3F2C9M11",
  "parsedResumeId": "pr_01J4QZ2Z1S5JZ9RZB9E1A2B8C3",
  "gates": [
    {"gateId": "gate_required_skills", "status": "PASS"},
    {"gateId": "gate_timezone_overlap", "status": "PASS"}
  ],
  "bucketScores": [
    {
      "bucketId": "hard_skills_tools",
      "score": 88,
      "weight": 0.60,
      "contribution": 52.8,
      "evidence": [
        {
          "type": "SKILL_MENTION",
          "skillId": "skill_sql",
          "confidence": 0.91,
          "matchType": "EXACT"
        }
      ],
      "missing": [
        {"skillId": "skill_sap_fi", "reason": "not evidenced within recency window"}
      ]
    }
  ],
  "modifiers": [
    {"modifierId": "data_fusion_confidence", "points": 1.2, "basis": {"avgNormConfidence": 0.82}},
    {"modifierId": "evidence_coverage_jd_duties", "points": 2.0, "basis": {"coveragePct": 0.74}}
  ],
  "metering": {
    "score_run_count": 1,
    "gemini_calls": 0,
    "db_writes": 6,
    "latency_ms_total": 187
  }
}

Response (DISQUALIFIED)

{
  "scorecardId": "sc_01J5S2M1P0K9N7B6C4D3E2F1A0",
  "status": "DISQUALIFIED",
  "scoreTotal": null,
  "baseFit": null,
  "modifierPoints": null,
  "disqualification": {
    "failedGates": [
      {
        "gateId": "gate_required_skills",
        "severity": "DISQUALIFY",
        "reason": "N_OF_M_SKILLS_NOT_MET",
        "details": {"minRequired": 4, "met": 2},
        "evidence": []
      }
    ]
  },
  "metering": {
    "score_run_count": 1,
    "gemini_calls": 0,
    "db_writes": 3,
    "latency_ms_total": 42
  }
}

Section 9: Enhancement Endpoints (All Delivered)

All enhancement endpoints are live on Cloud Run. They extend the core scoring with analytical and recruiter-facing capabilities.

Remaining (external dependencies):

• GitHub structured analysis: requires GitHub API integration
• LinkedIn enrichment via Proxycurl: requires API key procurement

Section 10: Scoring Runtime Workflow

10.1 Execution Lifecycle

1. INPUT RESOLUTION
   Load scoring config version (must be PUBLISHED)
   Load JD version + parsed resume + scoring input snapshot
   Validate required upstream gates passed

2. GATE EVALUATION (fail-fast)
   For each gate in config:
     Evaluate DSL rule against inputs
     If DISQUALIFY gate fails --> DISQUALIFIED (stop, persist, return)
     Record all gate results regardless of outcome

3. BUCKET SCORING (0..100 each)
   For each enabled bucket:
     Dispatch to ScorerRegistry (exact match on bucket_id)
     Compute deterministic score from evidence
     Record evidence_json, missing_json, rubric_applied_json

4. MODIFIER COMPUTATION
   For each modifier:
     Compute points from basis inputs
     Clamp by per-modifier min/max
   Sum all modifier points, clamp by global budget

5. ASSEMBLY
   base_fit = weighted sum of bucket scores
   score_total = clamp(base_fit + modifier_points, 0, 100)
   Map to band: STRONG / BORDERLINE / RISK

6. PERSISTENCE (transactional BEGIN/COMMIT)
   Insert scorecard_version
   Insert bucket_score[] rows
   Insert modifier_result[] rows
   Insert scoring_gate_result[] rows
   Insert scoring_run_metering
   Emit pipeline_phase_event (telemetry)

7. RETURN scorecard with full breakdown

10.2 Code Path

1. Scoring handoff: /internal/v1/scoring/build_inputs — takes parsed_resume_id or resolves from pipeline run. Creates scoring_input_snapshot.
2. Input snapshot: _ensure_scoring_input_snapshot() — loads parsed resume, pipeline run, page map, canonical extraction, normalization, gate status. Hard-blocks if required gates missing/failed.
3. Runtime entrypoint: /api/v1/scoring/scorecards — resolves published config, loads JD + parsed resume + scoring input, starts scoring run.
4. Gate evaluation: _evaluate_gate() for each gate in config.
5. Bucket dispatch: ScorerRegistry in src/scoring/runtime.py — exact match on bucket_id from config.
6. Modifier computation: Two heuristic modifiers: data fusion confidence and JD coverage ratio.
7. Persistence: Transactional BEGIN/COMMIT. Writes: scorecard_version, scoring_gate_result, bucket_score (with real rubric_applied_json), modifier_result, metering, audit events.

10.3 Idempotency

Every scoring operation that writes data checks idempotency_record first:

existing = await db.fetchrow(
    "SELECT id FROM intelletto.idempotency_record WHERE key = $1",
    idempotency_key
)
if existing:
    return {"status": "already_processed", "id": str(existing["id"])}

Silent double-writes corrupt the audit trail and are prevented by this guard.

10.4 Re-scoring

Re-scoring is append-only: it creates a new scorecard_version row. Prior scorecards are never overwritten. The re-scoring API supports:

• JD override (score against a different JD version)
• Config version override (use a different published scoring config)
• Single document: POST /api/v3/pipeline/rescore/{document_id}
• Bulk: POST /api/v3/pipeline/rescore/bulk

Section 11: Environment and Deployment

11.1 Technology Stack

• Backend: Python + FastAPI (async/await throughout)
• Database: PostgreSQL on Cloud SQL (asyncpg connection pool)
• AI: Google Gemini 2.5 Flash via Vertex AI for bounded inference only (extraction, normalization, soft signals). Always with response_schema enforcement. Never for scoring math.
• OCR: Google Document AI for layout parsing
• Storage: Google Cloud Storage for resume file ingestion and archival
• Orchestration: Netflix Conductor OSS — durable workflow execution with per-task retries, rate limits, timeouts, and HUMAN review gates
• Deployment: Cloud Run (API + Worker). Conductor on GCE.
• Secrets: Google Secret Manager (INTELLETTO_DB_URL, GEMINI_API_KEY, API_KEY)

Section 12: Known Gaps and P0 Status

12.1 What P0-3 Partial Means

The current system scores correctly using inline Python heuristics per bucket. Each scorer is registered in ScorerRegistry and dispatched by exact bucket_id match. The rubric_applied_json records the scorer_id and input/evidence hashes for reproducibility.

What is not yet implemented: loading rubric rules from the config_payload JSONB and applying them dynamically. Today, the rubric definitions in config are stored but the actual scoring logic is the inline Python implementation. This means changing scoring behavior requires code changes, not just config changes.

Do not attempt to close P0-3 without reading the full scoring spec and understanding the rubric DSL requirements.

Section 13: Non-Negotiable Invariants

1. Gate-first: Hard disqualifications run before scoring. If DISQUALIFIED, score_total = null and status = DISQUALIFIED. No bucket or modifier computation occurs.
2. Deterministic scoring: No stochastic scoring steps. Gemini only for bounded inference with schema validation and replayable inputs. Given the same (scoring_config_version_id, jd_version_id, parsed_resume_id), the system produces the same scorecard every time.
3. Append-only versioning: Re-scoring creates a new immutable scorecard version. Prior scorecards are never overwritten or deleted.
4. Evidence required: Every scored or disqualifying decision must carry evidence pointers (or explicit evidence_status = NONE_FOUND with search criteria recorded).
5. Compliance: Protected attributes (race, ethnicity, religion, age, sex, gender identity, health, disability, family status) are never used as scoring features. Prohibited feature list is enforced at ingestion and scoring runtime.
6. Config immutability: Published configs are frozen. Any change requires a new DRAFT version. Scoring runtime rejects non-PUBLISHED configs.
7. Idempotency: Every pipeline write checks idempotency_record first. Duplicate scoring requests return the existing scorecard without creating duplicate rows.
8. Audit trail: Every scoring run writes to pipeline_phase_event, scoring_run_metering, and creates immutable rows in all scoring tables. Failed runs are recorded, not silently swallowed.
9. Bucket weight invariant: Enabled bucket weights must sum to 1.0 (within tolerance of 0.0001). Config validation enforces this before publish.
10. Scorer registry contract: Bucket dispatch uses exact match on bucket_id via ScorerRegistry. No substring matching, no fallback logic.

13.1 Scoring System Safety Rules

• NEVER modify _score_bucket_* heuristic scorers without reading the full scoring spec
• NEVER change gate threshold values without reading the scoring config
• NEVER modify bucket weights without rubric alignment
• NEVER re-run scorecard generation for already-scored resumes without confirmation
• NEVER call /api/v1/scoring/scorecards on a resume that has a completed scorecard_version without using the re-score API

13.2 Lossless Resume Output Constraints

• NEVER produce: "No detailed achievements mapped in this parse." — this is always a defect
• NEVER truncate experience sections for any reason
• NEVER omit advisory roles, even those outside the main employment timeline
• NEVER omit education or certifications — if absent from source, write "Not captured in source document"
• NEVER treat normalization output as a replacement for source narrative content
• The Intelletto Resume word count must always exceed the source resume word count

Appendix A: Trial E2E Results (2026-04-02)

Full end-to-end pipeline validation with production data:

• 218 documents imported → 218/218 COMPLETED → 218/218 scored
• 0 failures, 0 zero-skill candidates
• All 12 pipeline stages ran for each document
• JD 59A356 (Senior Education Specialist): top score 79.8
• JD 479B1D (Sales Manager): top score 83.5

Concentrix demo dataset: 2,483 resumes across 24 AI-generated JDs, Filipino candidate profiles, full pipeline processing.

Version 1.0.0 — 2026-04-04 — Production System Reference