skelz0r/feature.md

## feature.md

      
    Raw
  

              feature.md
            
          
    The goal is to migrate all static models to the database.
We will start with the DataProvider model.
All of its attributes are defined in config/data_providers.yml.
The file format is:
slug:
    name: Name of the data provider
    logo: logo.png
    link: https://fournisseur-de-donnees.example.com
The attributes are:

name: the name of the data provider
logo: the path to the logo in the app/assets/images/data_providers folder
link: link to the data provider's website

We will use friendly_id as a unique attribute to identify each
data provider, based on the slug.
In terms of validation:

The slug must be unique, managed by friendly_id
The name must be present
The logo must be present
The link must be present and be a valid URL

For the logo, ActiveStorage will be used to manage the upload and
file association.
The .find will need to be modified throughout the code to use
.friendly.find instead to comply with slug management.
For data migration, a Rails migration will be created that will take the
YAML file and create the database records.

  
## final-plan.md

      
    Raw
  

              final-plan.md
            
          
    Migration Plan: DataProvider from YAML to Database

Context

The DataProvider model is currently a static model (inheriting from StaticApplicationRecord) that loads its data from config/data_providers.yml. The goal is to migrate it to a database-backed model with the following changes:

Convert from StaticApplicationRecord to ApplicationRecord (database-backed)
Use friendly_id gem for slug-based lookups (slug derived from current YAML keys for import, auto-generated from name for future records)
Use ActiveStorage for logo file management (same service as other uploads)
Add validations for name, logo, and link (URL validation using same regex as cadre_juridique)
Migrate existing YAML data to database records
Update all code references from .find(id) to .friendly.find(slug)
Update view helpers to use ActiveStorage URLs instead of asset pipeline
Keep YAML file for reference/backup (no deletion)

Important: This is a backend-only migration. No controllers or admin UI will be created. Future management will be via Rails console.
Current State Analysis

Existing Data (12 providers in YAML)


dgs - Direction Générale de la Santé
dila - Direction de l'Information Légale et Administrative
dinum - DINUM
dgfip - Direction Générale des Finances Publiques
ministere_des_armees - Ministère Des Armées
aife - Agence pour l'Information Financière de l'État
ans - Agence du Numérique en Santé
urssaf - URSSAF
menj - Ministère de l'Éducation Nationale et de la Jeunesse
cnam - CNAM
cisirh - Centre Interministériel des Systèmes d'Information
mtes - Ministère de la Transition écologique

Code Locations Using DataProvider


app/models/authorization_definition.rb:47 - Loads provider from YAML hash
app/controllers/dgfip/export_controller.rb:38 - Finds 'dgfip' provider
app/helpers/application_helper.rb:19 - Displays provider logo using asset pipeline

Existing Relationships


AuthorizationDefinition: In-memory relationship via provider.id matching
User: Indirect relationship through authorization_definitions (reporters/instructors methods)

Technology Stack


Rails 8.0 / Ruby 3.4.1
PostgreSQL (with hstore, plpgsql, pgcrypto extensions)
friendly_id gem ~> 5.5.0 (already installed)
ActiveStorage (already configured, uses S3 in production)
active_storage_validations gem (already installed)

Implementation Plan

Phase 1: Model and Database Setup

Step 1.1: Create Database Migration

File: db/migrate/YYYYMMDDHHMMSS_create_data_providers.rb
Create table with:
create_table :data_providers do |t|
  t.string :slug, null: false, index: { unique: true }
  t.string :name, null: false
  t.string :link, null: false

  t.timestamps
end
Notes:

Logo will be handled by ActiveStorage (no column needed)
Slug will be used for friendly_id lookups
Index on slug for performance

Test: Run migration in test environment
RAILS_ENV=test bundle exec rails db:migrate
Step 1.2: Copy Logo to Fixtures

File: spec/fixtures/files/dinum.png
Copy an existing logo to fixtures for testing:
mkdir -p spec/fixtures/files
cp app/assets/images/data_providers/dinum.png spec/fixtures/files/dinum.png
Notes:

Use dinum.png as the test fixture logo (exists in assets)
Fixtures directory at spec/fixtures/files/ (Rails convention)
This allows tests to attach real image files

Step 1.3: Create FactoryBot Factory

File: spec/factories/data_providers.rb
Create factory:
FactoryBot.define do
  sequence(:data_provider_slug) { |n| "provider_#{n}" }

  factory :data_provider do
    slug { generate(:data_provider_slug) }
    name { 'Test Provider' }
    link { 'https://test-provider.gouv.fr' }

    after(:build) do |data_provider|
      data_provider.logo.attach(
        io: File.open(Rails.root.join('spec', 'fixtures', 'files', 'dinum.png')),
        filename: 'dinum.png',
        content_type: 'image/png'
      )
    end

    trait :dgfip do
      slug { 'dgfip' }
      name { 'DGFIP' }
      link { 'https://api.gouv.fr/producteurs/dgfip' }
    end

    trait :dinum do
      slug { 'dinum' }
      name { 'DINUM' }
      link { 'https://www.numerique.gouv.fr/' }
    end
  end
end
Notes:

Uses real logo file from fixtures (dinum.png)
Opens file directly instead of using StringIO
Create traits for commonly used providers (dgfip, dinum) for tests

Test: Factory can be used in specs
Step 1.4: Update DataProvider Model

File: app/models/data_provider.rb
Complete rewrite:
class DataProvider < ApplicationRecord
  extend FriendlyId

  URL_REGEX = %r{\A((http|https)://)?[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}(/[\w\-._~:/?#\[\]@!$&'()*+,;%=]*)?\z}

  friendly_id :slug, use: :slugged

  has_one_attached :logo

  validates :slug, presence: true, uniqueness: true
  validates :name, presence: true
  validates :link, presence: true, format: { with: URL_REGEX, message: I18n.t('activemodel.errors.messages.url_format') }
  validates :logo, attached: true, content_type: ['image/png', 'image/jpg', 'image/jpeg']

  def authorization_definitions
    @authorization_definitions ||= AuthorizationDefinition.all.select do |authorization_definition|
      authorization_definition.provider.id == id.to_s
    end
  end

  def reporters
    users_for_roles(%w[instructor reporter])
  end

  def instructors
    users_for_roles(%w[instructor])
  end

  private

  def users_for_roles(roles)
    User.where(
      "EXISTS (
        SELECT 1
        FROM unnest(roles) AS role
        WHERE role in (?)
      )",
      roles.map { |role| build_user_role_query_param(role) }.flatten,
    )
  end

  def build_user_role_query_param(role)
    authorization_definitions.map do |authorization_definition|
      "#{authorization_definition.id}:#{role}"
    end
  end
end
Key changes:

Inherit from ApplicationRecord (not StaticApplicationRecord)
Add friendly_id configuration (slug-based, like Authorization model)
Add has_one_attached :logo for ActiveStorage
Add validations (name, slug, link format, logo attachment)
Keep existing methods: authorization_definitions, reporters, instructors, users_for_roles, build_user_role_query_param
Remove: attr_accessor, self.backend method
Update authorization_definitions to compare with id.to_s (database id vs static string)

Notes:

URL regex matches pattern from app/models/concerns/authorization_extensions/cadre_juridique.rb
ActiveStorage validations use active_storage_validations gem
Logo content types match existing files (png, jpg, jpeg)

Test: Model tests (see Step 1.5)
Step 1.5: Write Model Tests

File: spec/models/data_provider_spec.rb
Update/add tests:
RSpec.describe DataProvider do
  describe 'validations' do
    subject { build(:data_provider) }

    it { is_expected.to validate_presence_of(:slug) }
    it { is_expected.to validate_presence_of(:name) }
    it { is_expected.to validate_presence_of(:link) }
    it { is_expected.to validate_uniqueness_of(:slug) }

    describe 'link format validation' do
      it 'accepts valid URLs' do
        valid_urls = [
          'https://www.example.gouv.fr',
          'http://example.gouv.fr',
          'https://example.gouv.fr/path/to/page',
        ]

        valid_urls.each do |url|
          subject.link = url
          expect(subject).to be_valid, "Expected #{url} to be valid"
        end
      end

      it 'rejects invalid URLs' do
        invalid_urls = ['not a url', 'ftp://example.com', 'example', '']

        invalid_urls.each do |url|
          subject.link = url
          expect(subject).not_to be_valid, "Expected #{url} to be invalid"
        end
      end
    end

    describe 'logo attachment' do
      it 'validates logo is attached' do
        provider = build(:data_provider)
        provider.logo.purge
        expect(provider).not_to be_valid
        expect(provider.errors[:logo]).to be_present
      end

      it 'validates logo content type' do
        provider = build(:data_provider)
        provider.logo.attach(io: StringIO.new('content'), filename: 'test.txt', content_type: 'text/plain')
        expect(provider).not_to be_valid
      end

      it 'accepts valid image types' do
        %w[image/png image/jpg image/jpeg].each do |content_type|
          provider = build(:data_provider)
          provider.logo.attach(io: StringIO.new('content'), filename: 'test.png', content_type:)
          expect(provider).to be_valid, "Expected #{content_type} to be valid"
        end
      end
    end
  end

  describe 'friendly_id' do
    it 'finds by slug using friendly.find' do
      provider = create(:data_provider, slug: 'test-provider')
      expect(DataProvider.friendly.find('test-provider')).to eq(provider)
    end

    it 'raises error when slug not found' do
      expect { DataProvider.friendly.find('nonexistent') }.to raise_error(ActiveRecord::RecordNotFound)
    end
  end

  describe '#authorization_definitions' do
    it 'returns authorization definitions for this provider' do
      provider = create(:data_provider, slug: 'dgfip')

      definitions = provider.authorization_definitions

      expect(definitions).to be_all { |d| d.provider.id == 'dgfip' }
    end
  end

  describe '#reporters' do
    subject { create(:data_provider, :dgfip).reporters }

    let!(:valid_users) do
      [
        create(:user, :reporter, authorization_request_types: %w[api_impot_particulier api_hermes]),
        create(:user, :instructor, authorization_request_types: %w[api_hermes]),
      ]
    end

    let!(:invalid_users) do
      [
        create(:user, :reporter, authorization_request_types: %w[api_entreprise]),
      ]
    end

    it { is_expected.to match_array(valid_users) }
  end

  describe '#instructors' do
    subject { create(:data_provider, :dgfip).instructors }

    let!(:valid_users) do
      [
        create(:user, :instructor, authorization_request_types: %w[api_impot_particulier api_hermes]),
        create(:user, :instructor, authorization_request_types: %w[api_hermes]),
      ]
    end

    let!(:invalid_users) do
      [
        create(:user, :reporter, authorization_request_types: %w[api_hermes]),
        create(:user, :reporter, authorization_request_types: %w[api_entreprise]),
      ]
    end

    it { is_expected.to match_array(valid_users) }
  end
end
Notes:

Reuse existing test patterns from current spec/models/data_provider_spec.rb
Add validation tests for all new validations
Test friendly_id functionality
Keep existing reporters and instructors tests (should still work)
Use factory with traits for common providers

Test: Run specs
bundle exec rspec spec/models/data_provider_spec.rb
Phase 2: Data Migration

Step 2.1: Create Data Migration

File: db/migrate/YYYYMMDDHHMMSS_migrate_data_providers_from_yaml.rb
Migration to import YAML data with strict validation:
class MigrateDataProvidersFromYaml < ActiveRecord::Migration[8.0]
  def up
    providers_data = Rails.application.config_for(:data_providers)

    providers_data.each do |slug, attributes|
      provider = DataProvider.create!(
        slug:,
        name: attributes['name'],
        link: attributes['link']
      )

      logo_filename = attributes['logo']
      logo_path = Rails.root.join('app', 'assets', 'images', 'data_providers', logo_filename)

      extension = File.extname(logo_filename).downcase

      unless File.exist?(logo_path)
        raise "Logo file not found for provider '#{slug}': #{logo_path}"
      end

      unless ['.png', '.jpg', '.jpeg'].include?(extension)
        raise "Invalid logo format for provider '#{slug}': #{logo_filename}. Only PNG and JPG/JPEG are allowed."
      end

      content_type = case extension
                     when '.png' then 'image/png'
                     when '.jpg', '.jpeg' then 'image/jpeg'
                     end

      provider.logo.attach!(
        io: File.open(logo_path),
        filename: logo_filename,
        content_type:
      )

      Rails.logger.info "Migrated provider '#{slug}' with logo '#{logo_filename}'"
    end

    Rails.logger.info "Successfully migrated #{providers_data.count} data providers"
  end

  def down
    DataProvider.destroy_all
  end
end
Key changes from previous version:

Use attach! instead of attach (raises error on failure)
Raise explicit error if logo file doesn't exist (no silent warnings)
Validate file extension before attempting attach (only .png, .jpg, .jpeg)
Raise error for invalid file formats
Add info logging for successful migrations
Migration will fail fast on any error

Notes:

Migration will halt deployment if any logo is missing or invalid
All validations happen before database write
Reversible migration (down method destroys all records)

Test: Run migration in test environment
RAILS_ENV=test bundle exec rails db:migrate
Verify:
RAILS_ENV=test bundle exec rails runner "puts DataProvider.count" # Should be 12
RAILS_ENV=test bundle exec rails runner "puts DataProvider.all.map(&:slug).join(', ')"
Step 2.2: Verify Data Integrity

Manual verification after migration:
RAILS_ENV=test bundle exec rails runner "
  DataProvider.all.each do |p|
    puts '#{p.slug}: #{p.name}, logo: #{p.logo.attached?}, link: #{p.link}'
  end
"
Check:

All 12 providers created
All slugs match YAML keys (dgs, dila, dinum, dgfip, etc.)
All logos attached
All links populated

Phase 3: Update Code References

Step 3.1: Update AuthorizationDefinition

File: app/models/authorization_definition.rb:47
Change:
# Before
provider: DataProvider.find(hash[:provider]),

# After
provider: DataProvider.friendly.find(hash[:provider]),
Notes:

Most critical change - used during AuthorizationDefinition loading
AuthorizationDefinition is still a StaticApplicationRecord loading from YAML
The provider slug in authorization_definitions YAML must match DataProvider slugs

Test: Verify authorization definitions load correctly
RAILS_ENV=test bundle exec rails runner "puts AuthorizationDefinition.all.count"
Step 3.2: Update DGFIP Export Controller

File: app/controllers/dgfip/export_controller.rb:38
Change:
# Before
@data_provider ||= DataProvider.find('dgfip')

# After
@data_provider ||= DataProvider.friendly.find('dgfip')
Test: Verify controller (if specs exist)
Step 3.3: Update Application Helper for Logo Display

File: app/helpers/application_helper.rb:17-20
Change (simplified - logo is always present):
# Before
def provider_logo_image_tag(authorization_definition, options = {})
  options = options.merge(alt: "Logo du fournisseur de données \" #{authorization_definition.provider.name}\"")
  image_tag("data_providers/#{authorization_definition.provider.logo}", options)
end

# After
def provider_logo_image_tag(authorization_definition, options = {})
  options = options.merge(alt: "Logo du fournisseur de données \" #{authorization_definition.provider.name}\"")
  image_tag(authorization_definition.provider.logo, options)
end
Notes:

Use ActiveStorage's image_tag helper (works directly with attachments)
No fallback needed - logo is always present after migration (validated)
Simpler implementation since validation ensures logo exists

Test: Manual testing in views that use provider logos
Phase 4: Testing and Validation

Step 4.1: Run All RSpec Tests

Command:
bundle exec rspec
Verify:

DataProvider model tests pass
All related model tests pass (AuthorizationDefinition, User)
Controller tests pass (if any for dgfip export)
Helper tests pass (if any)
No regressions in other tests

Fix failures as they occur
Step 4.2: Run Cucumber Features

Command:
bundle exec cucumber
Verify:

All E2E tests pass
Features using DataProvider work correctly
Logo display works in views

Fix failures as they occur
Step 4.3: Manual Testing in Development

Start server:
bin/local_run.sh
Test:


Check DataProvider records exist:
DataProvider.count # Should be 12
DataProvider.all.map(&:slug)


Test friendly_id lookups:
DataProvider.friendly.find('dgfip')
DataProvider.friendly.find('dinum')


Check logos attached:
DataProvider.all.each { |p| puts "#{p.slug}: #{p.logo.attached?}" }


Test relationships:
dgfip = DataProvider.friendly.find('dgfip')
dgfip.authorization_definitions.map(&:id)
dgfip.reporters.count
dgfip.instructors.count


Browse pages that display provider logos


Test DGFIP export functionality (if accessible)


Step 4.4: Run Linters

Commands:
bundle exec rubocop
bundle exec rubocop -A  # Auto-fix issues
Fix any style violations:

Maximum method length: 15 lines
Class length: max 150 lines
Use single quotes for strings
2 spaces indentation

Phase 5: Documentation

Step 5.1: Add Comment to YAML File

File: config/data_providers.yml
Add comment at the top:
---
# NOTE: This file is kept for reference purposes only.
# Data providers are now stored in the database (data_providers table).
# To add new providers, use the Rails console:
#   provider = DataProvider.create!(
#     name: 'Provider Name',
#     link: 'https://provider.gouv.fr'
#   )
#   provider.logo.attach(io: File.open('path/to/logo.png'), filename: 'logo.png', content_type: 'image/png')
#
# This file was used for the initial data migration.
# Migration date: [INSERT DATE]

shared:
  dgs:
    name: Organisation de la direction générale de la santé (DGS)
    logo: ministere_sante.jpg
    link: https://www.sante.gouv.fr/
  # ... rest of file
Notes:

Keep YAML file as backup/reference (per user decision)
Add clear instructions for future additions via console
Document migration date

Step 5.2: Update Technical Documentation (if exists)

Check these files:
ls docs/
Update if DataProvider is mentioned:

Architecture diagrams
Data model documentation
Development guides

Document:

DataProvider is now database-backed
Uses friendly_id for slug-based lookups
Uses ActiveStorage for logo management
How to add new providers via console


Testing Strategy

Following TDD approach per repository guidelines:
Step 1: Models


Create migration
Copy logo to fixtures
Create factory
Write failing model tests
Implement model changes
Run tests until passing
Run rubocop and fix issues
Refactor if needed

Step 2: Data Migration


Write migration with strict validation
Test in test environment
Verify data integrity
Test idempotency (run twice, check no duplicates)
Test rollback (down migration)

Step 3: Code Updates


Update each file one at a time
Run related tests after each change
Fix failures
Run rubocop

Step 4: Integration


Run full RSpec suite
Run full Cucumber suite
Manual testing in development
Fix any issues


Rollback Strategy

If migration fails in production:


During deployment: Deployment will fail and rollback automatically (per user decision)

Migration uses attach! which raises errors
Any missing or invalid logo will halt migration
Database transaction ensures atomicity


After deployment:

Run down migration: rails db:rollback STEP=2 (2 migrations: data + table)
YAML file still exists as backup
Code will need to be reverted via git


Data recovery:

YAML file remains untouched
Can re-import from YAML if needed
Asset logos remain in app/assets/images/data_providers/


Success Criteria


 DataProvider table created with all required columns and indexes
 Logo fixture copied to spec/fixtures/files/
 Factory created for testing with real logo file
 All 12 existing providers migrated to database with correct data
 All logos attached via ActiveStorage and display correctly
 All validations working (name, logo, link presence and format)
 friendly_id working for slug-based lookups
 All code references updated to use .friendly.find()
 Application helper updated to use ActiveStorage URLs (simplified, no fallback)
 All existing functionality preserved (authorization_definitions, reporters, instructors)
 All RSpec tests passing
 All Cucumber features passing
 Rubocop passing with no violations
 No regressions in application behavior
 YAML file kept with documentation comment
 Manual testing completed successfully


Estimated Implementation Order


Create table migration (30 min)
Copy logo to fixtures (5 min)
Create FactoryBot factory with real logo (30 min)
Update DataProvider model (1 hour)
Write model tests (1.5 hours)
Run tests and fix issues (1 hour)
Create data migration script with strict validation (1 hour)
Test data migration in test environment (30 min)
Update AuthorizationDefinition (15 min)
Update DGFIP export controller (15 min)
Update application helper (15 min - simplified)
Run all RSpec tests (30 min)
Fix any failing tests (1 hour buffer)
Run Cucumber features (30 min)
Fix any failing features (30 min)
Run rubocop and fix style issues (30 min)
Manual testing (1 hour)
Add YAML comment and documentation (30 min)

Total estimated time: ~11.5 hours

Risk Assessment

High Risk


Data migration integrity: Losing or corrupting provider data

Mitigation: Test thoroughly in test environment first, keep YAML backup, reversible migration, use attach! to fail fast


Breaking AuthorizationDefinition loading: Critical part of the app

Mitigation: Extensive testing, verify all definitions load correctly


Logo display breaking: Views depend on logo display

Mitigation: Test helper thoroughly, validation ensures logo always present


Medium Risk


Missing logo files: Some logos might not exist or have wrong filenames

Mitigation: Migration raises explicit errors for missing files, strict validation before attach


URL validation too strict: Existing URLs might not match regex

Mitigation: Test regex against all existing URLs from YAML first


ID comparison issues: Database IDs (integers) vs YAML IDs (strings)

Mitigation: Use .to_s in comparisons, test authorization_definitions relationship thoroughly


Low Risk


Performance: Database queries instead of in-memory lookups

Mitigation: Add database indexes on slug, use Rails query cache


Slug conflicts: Unlikely with current data

Mitigation: friendly_id handles this with candidate generation


Notes


This migration follows the same pattern as the Authorization model's use of friendly_id
No controllers or views will be created (backend-only migration)
Future DataProvider management via Rails console only
YAML file kept as backup per user decision
StaticApplicationRecord class remains (other models still use it)
Uses same ActiveStorage service as rest of application
All existing functionality (reporters, instructors, authorization_definitions) preserved
Logo display updated to use ActiveStorage instead of asset pipeline
Migration uses strict validation with attach! to ensure data integrity
No fallback needed in helper - logo is always validated to be present


Open Questions

None - all questions have been answered by the user and all FIXME comments addressed.

  
## plan-1.md

      
    Raw
  

              plan-1.md
            
          
    Migration Plan: DataProvider from YAML to Database

Context

The DataProvider model is currently a static model (inheriting from StaticApplicationRecord) that loads its data from config/data_providers.yml. The goal is to migrate it to a database-backed model with the following changes:

Convert from StaticApplicationRecord to ApplicationRecord (database-backed)
Use friendly_id gem for slug-based lookups (slug derived from current YAML keys)
Use ActiveStorage for logo file management
Add validations for name, logo, and link (URL validation)
Migrate existing YAML data to database records
Update all code references from .find(id) to .friendly.find(slug)

Current State Analysis

Existing Data (12 providers in YAML)


dgs - Direction Générale de la Santé
dila - Direction de l'Information Légale et Administrative
dinum - DINUM
dgfip - Direction Générale des Finances Publiques
ministere_des_armees - Ministère Des Armées
aife - Agence pour l'Information Financière de l'État
ans - Agence du Numérique en Santé
urssaf - URSSAF
menj - Ministère de l'Éducation Nationale et de la Jeunesse
cnam - CNAM
cisirh - Centre Interministériel des Systèmes d'Information
mtes - Ministère de la Transition écologique

Code Locations Using DataProvider.find


app/models/authorization_definition.rb:47 - Loads provider from YAML hash
app/controllers/dgfip/export_controller.rb:38 - Finds 'dgfip' provider

Existing Relationships


AuthorizationDefinition: In-memory relationship via provider.id matching
User: Indirect relationship through authorization_definitions (reporters/instructors methods)

Implementation Plan

Phase 1: Model and Database Setup

Step 1.1: Create Database Migration

File: db/migrate/YYYYMMDDHHMMSS_create_data_providers.rb
Create table with:

id (bigint, primary key)
slug (string, unique, not null, indexed)
name (string, not null)
link (string, not null)
timestamps (created_at, updated_at)

Notes:

Logo will be handled by ActiveStorage (no column needed)
Slug will be used for friendly_id lookups

Test: Run migration in test environment to verify table creation
Step 1.2: Update DataProvider Model

File: app/models/data_provider.rb
Changes:

Inherit from ApplicationRecord instead of StaticApplicationRecord
Add friendly_id configuration:
extend FriendlyId
friendly_id :slug, use: :slugged

Add has_one_attached :logo for ActiveStorage
Add validations:

validates :slug, presence: true, uniqueness: true
validates :name, presence: true
validates :logo, presence: true (using active_storage_validations)
validates :link, presence: true, format: { with: URL_REGEX } (similar pattern to cadre_juridique_url)


Keep existing methods:

authorization_definitions
reporters
instructors
users_for_roles (private)
build_user_role_query_param (private)


Remove:

attr_accessor declarations (replaced by database columns)
self.backend class method (no longer needed)


Notes:

The URL regex can be inspired from app/models/concerns/authorization_extensions/cadre_juridique.rb:13
For ActiveStorage validation, check if active_storage_validations gem is available

Test: Create RSpec tests for the new model (see Step 1.3)
Step 1.3: Write Model Tests

File: spec/models/data_provider_spec.rb
Update existing tests and add new ones:

Test validations:

Presence of slug, name, logo, link
Uniqueness of slug
URL format validation for link
Logo attachment presence


Test friendly_id:

Can find by slug using .friendly.find(slug)
Slug is auto-generated if not provided (from id or name)


Test relationships:

authorization_definitions returns correct definitions
reporters returns correct users
instructors returns correct users


Test behavior:

Can attach logo via ActiveStorage
Logo is persisted correctly


Notes:

Use FactoryBot to create test data providers with logos
May need to create a factory for DataProvider with logo attachment

Test: Run bundle exec rspec spec/models/data_provider_spec.rb
Phase 2: Data Migration

Step 2.1: Create Data Migration

File: db/migrate/YYYYMMDDHHMMSS_migrate_data_providers_from_yaml.rb
Migration logic:

Read config/data_providers.yml using Rails.application.config_for(:data_providers)
For each entry (slug, attributes):

Create DataProvider record with slug and attributes
Attach logo file from app/assets/images/data_providers/#{logo_filename}
Handle file reading and attachment via ActiveStorage


Notes:

Use up method (not change) to allow custom logic
Include error handling for missing logo files
Use File.open to read logo files
Attach using logo.attach(io: file, filename: filename, content_type: content_type)
Can reference the pattern from other data migrations like 20240906162647_create_bulk_update_for_api_particulier_on_contact_metier.rb

Test: Run migration in test environment and verify:

All 12 providers are created
Slugs match YAML keys
Logos are attached correctly
All attributes are populated

Step 2.2: Create Migration Test

File: spec/migrations/migrate_data_providers_from_yaml_spec.rb (or similar)
Test:

Migration creates all expected providers
Slugs match YAML keys
Logos are attached
All attributes are correct

Notes:

May not be necessary if we manually verify the migration
Can be skipped if project doesn't have migration tests

Phase 3: Update Code References

Step 3.1: Update AuthorizationDefinition

File: app/models/authorization_definition.rb:47
Change:
# Before
provider: DataProvider.find(hash[:provider]),

# After
provider: DataProvider.friendly.find(hash[:provider]),
Notes:

This is the most critical change as it's used in the static model loading
AuthorizationDefinition is also a StaticApplicationRecord, so it still loads from YAML
The provider lookup needs to work with the database-backed model

Test: Verify AuthorizationDefinition loading still works
Step 3.2: Update DGFIP Export Controller

File: app/controllers/dgfip/export_controller.rb:38
Change:
# Before
@data_provider ||= DataProvider.find('dgfip')

# After
@data_provider ||= DataProvider.friendly.find('dgfip')
Test: Test the export controller action
Phase 4: Testing and Validation

Step 4.1: Run All RSpec Tests

Command: bundle exec rspec
Verify:

DataProvider model tests pass
All related model tests pass
No regressions in other tests

Step 4.2: Run Cucumber Features

Command: bundle exec cucumber
Verify:

All E2E tests pass
Features using DataProvider work correctly

Step 4.3: Manual Testing

Test:

Load application in development
Verify DataProvider records exist in database
Check that logos display correctly
Verify authorization_definitions relationship works
Test reporter/instructor queries
Check DGFIP export functionality (if applicable)

Step 4.4: Run Linters

Commands:

bundle exec rubocop (Ruby linting)
standard app/javascript (JavaScript linting, if applicable)

Fix any style violations:

bundle exec rubocop -A (auto-fix)

Phase 5: Cleanup (Optional)

Step 5.1: Archive or Remove YAML File

FIXME pas besoin on le garde
File: config/data_providers.yml
Options:

Keep file for reference (add comment about migration)
Move to config/archived/data_providers.yml
Delete file (data now in database)

Notes:

Discuss with team before removing
May want to keep for rollback purposes

Step 5.2: Remove StaticApplicationRecord (Future Work)

FIXME non
If DataProvider was the last model using StaticApplicationRecord:

Consider removing app/models/static_application_record.rb
Update other static models as well (ServiceProvider, IdentityProvider, etc.)

Notes:

This is out of scope for current task
Document as future improvement

Phase 6: Documentation

Step 6.1: Update Technical Documentation

Check if any docs need updating:

docs/ folder files
README (if mentions DataProvider)
Architecture documentation

Step 6.2: Add Migration Notes

Document:

Why migration was done
How to add new data providers (now via admin interface or console)
Logo upload process


Testing Strategy

Following TDD approach:


Step 1: Models

Write failing tests for new DataProvider model
Implement model to make tests pass
Refactor and ensure rubocop passes


Step 2: Data Migration

Test migration in test environment
Verify data integrity
Ensure idempotency (can run multiple times safely)


Step 3: Code Updates

Write/update tests for each changed file
Implement changes
Verify tests pass


Step 4: Integration

Run full test suite
Run cucumber features
Manual testing in development environment


Questions for Clarification


Slug Generation: Should the slug be the same as the current YAML keys (dgs, dila, dgfip, etc.), or should it be auto-generated from the name?

Recommendation: Use current YAML keys as slugs for backward compatibility


Oui pour l'import, et pour les créations futures on se base sur le nom.

Logo Management: After migration, how should new logos be added?

Option A: Via admin interface (requires building admin UI)
Option B: Via Rails console
Option C: Keep ability to seed from assets folder


On verra plus tard.

YAML File: Should we keep the YAML file after migration?

Option A: Keep for reference/rollback
Option B: Archive in config/archived/
Option C: Delete entirely


Option A.

Authorization: Who should be able to create/update/delete DataProviders?

Currently no authorization mentioned
Should only admins manage this?
Should we add a basic admin interface?


Purement une migration de backend, aucun controller/vues prévues.

Rollback Strategy: If migration fails in production, what's the rollback plan?

Keep YAML file as backup?
Migration down method to restore StaticApplicationRecord pattern?


On fail pour annuler le déploiement.

Other Static Models: Are we planning to migrate other static models (ServiceProvider, IdentityProvider, Subdomain, CodeNAF)?

Should we establish a pattern for all static models?
Or is this a one-off migration?


Non on verra plus tard.

Logo Display: How are logos currently displayed in views?

Are they using asset pipeline (image_tag)?
Will need to update to use ActiveStorage URLs
Should we add an image variant/processing?


Oui il faut utiliser les méthodes classiques d'ActiveStorage pour l'affichage.

Friendly ID Finders: The codebase currently uses .friendly.find() for Authorization model, but the friendly_id config has config.use :finders disabled. Do we need to enable it or stick with explicit .friendly.find()?

Recommendation: Use explicit .friendly.find() to match existing pattern


Oui garde friendly

URL Validation: Should we use the same URL regex pattern as in cadre_juridique.rb, or is there a preferred URL validation gem/method?

Current pattern: %r{\A((http|https)://)?[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}(/[\w\-._~:/?#\[\]@!$&'()*+,;%=]*)?\z}


Oui très bien

ActiveStorage Service: Should logos use the same ActiveStorage service as other uploads (likely S3 in production)?

Or should they use local storage since they're relatively static?


On garde le même service que le reste.

Success Criteria


 DataProvider table created with all required columns and indexes
 All 12 existing providers migrated to database with correct data
 All logos attached via ActiveStorage and display correctly
 All validations working (name, logo, link presence and format)
 friendly_id working for slug-based lookups
 All code references updated to use .friendly.find()
 All existing functionality preserved (authorization_definitions, reporters, instructors)
 All RSpec tests passing
 All Cucumber features passing
 Rubocop passing with no violations
 No regressions in application behavior


Estimated Implementation Order


✅ Create table migration (30 min)
✅ Update DataProvider model (1 hour)
✅ Write model tests (1 hour)
✅ Run tests and fix issues (30 min)
✅ Create data migration script (1 hour)
✅ Test data migration (30 min)
✅ Update code references (30 min)
✅ Run full test suite (30 min)
✅ Fix any failing tests (1 hour buffer)
✅ Run rubocop and fix style issues (30 min)
✅ Manual testing (1 hour)
✅ Documentation updates (30 min)

Total estimated time: ~9 hours

Risk Assessment

High Risk


Data migration integrity: Losing or corrupting provider data

Mitigation: Test thoroughly in test environment first, keep YAML backup


Breaking AuthorizationDefinition loading: This is a critical part of the app

Mitigation: Extensive testing of authorization definition loading


Medium Risk


Missing logo files: Some logos might not exist or have wrong filenames

Mitigation: Verify all logo files exist before migration, add error handling


URL validation too strict: Existing URLs might not match regex

Mitigation: Test regex against all existing URLs first


Low Risk


Performance: Database queries instead of in-memory lookups

Mitigation: Add database indexes, use caching if needed


Slug conflicts: Unlikely with current data, but possible

Mitigation: friendly_id handles this automatically with candidate generation


Notes


This migration follows the same pattern as the Authorization model's use of friendly_id
The codebase uses interactor gem for services, but this migration might not need a complex service
Consider if DataProvider needs a decorator (Draper) for view-related logic
Current logo path pattern: app/assets/images/data_providers/#{filename}
All logos are in different formats: jpg, jpeg, png


## plan-2.md

      
    Raw
  

              plan-2.md
            
          
    Migration Plan: DataProvider from YAML to Database

Context

The DataProvider model is currently a static model (inheriting from StaticApplicationRecord) that loads its data from config/data_providers.yml. The goal is to migrate it to a database-backed model with the following changes:

Convert from StaticApplicationRecord to ApplicationRecord (database-backed)
Use friendly_id gem for slug-based lookups (slug derived from current YAML keys for import, auto-generated from name for future records)
Use ActiveStorage for logo file management (same service as other uploads)
Add validations for name, logo, and link (URL validation using same regex as cadre_juridique)
Migrate existing YAML data to database records
Update all code references from .find(id) to .friendly.find(slug)
Update view helpers to use ActiveStorage URLs instead of asset pipeline
Keep YAML file for reference/backup (no deletion)

Important: This is a backend-only migration. No controllers or admin UI will be created. Future management will be via Rails console.
Current State Analysis

Existing Data (12 providers in YAML)


dgs - Direction Générale de la Santé
dila - Direction de l'Information Légale et Administrative
dinum - DINUM
dgfip - Direction Générale des Finances Publiques
ministere_des_armees - Ministère Des Armées
aife - Agence pour l'Information Financière de l'État
ans - Agence du Numérique en Santé
urssaf - URSSAF
menj - Ministère de l'Éducation Nationale et de la Jeunesse
cnam - CNAM
cisirh - Centre Interministériel des Systèmes d'Information
mtes - Ministère de la Transition écologique

Code Locations Using DataProvider


app/models/authorization_definition.rb:47 - Loads provider from YAML hash
app/controllers/dgfip/export_controller.rb:38 - Finds 'dgfip' provider
app/helpers/application_helper.rb:19 - Displays provider logo using asset pipeline

Existing Relationships


AuthorizationDefinition: In-memory relationship via provider.id matching
User: Indirect relationship through authorization_definitions (reporters/instructors methods)

Technology Stack


Rails 8.0 / Ruby 3.4.1
PostgreSQL (with hstore, plpgsql, pgcrypto extensions)
friendly_id gem ~> 5.5.0 (already installed)
ActiveStorage (already configured, uses S3 in production)
active_storage_validations gem (already installed)

Implementation Plan

Phase 1: Model and Database Setup

Step 1.1: Create Database Migration

File: db/migrate/YYYYMMDDHHMMSS_create_data_providers.rb
Create table with:
create_table :data_providers do |t|
  t.string :slug, null: false, index: { unique: true }
  t.string :name, null: false
  t.string :link, null: false

  t.timestamps
end
Notes:

Logo will be handled by ActiveStorage (no column needed)
Slug will be used for friendly_id lookups
Index on slug for performance

Test: Run migration in test environment
RAILS_ENV=test bundle exec rails db:migrate
Step 1.2: Create FactoryBot Factory

File: spec/factories/data_providers.rb
FIXME ajoute un faux logo dans les fixtures, prends en un dans
app/assets/images/data_providers/
Create factory:
FactoryBot.define do
  sequence(:data_provider_slug) { |n| "provider_#{n}" }

  factory :data_provider do
    slug { generate(:data_provider_slug) }
    name { 'Test Provider' }
    link { 'https://test-provider.gouv.fr' }

    transient do
      logo_filename { 'test-logo.png' }
      logo_content { 'fake_logo_content' }
    end

    after(:build) do |data_provider, evaluator|
      data_provider.logo.attach(
        io: StringIO.new(evaluator.logo_content),
        filename: evaluator.logo_filename,
        content_type: 'image/png'
      )
    end

    trait :dgfip do
      slug { 'dgfip' }
      name { 'DGFIP' }
      link { 'https://api.gouv.fr/producteurs/dgfip' }
    end

    trait :dinum do
      slug { 'dinum' }
      name { 'DINUM' }
      link { 'https://www.numerique.gouv.fr/' }
    end
  end
end
Notes:

Use pattern from spec/factories/active_storage_blob.rb
Create traits for commonly used providers (dgfip, dinum) for tests

Test: Factory can be used in specs
Step 1.3: Update DataProvider Model

File: app/models/data_provider.rb
Complete rewrite:
class DataProvider < ApplicationRecord
  extend FriendlyId

  URL_REGEX = %r{\A((http|https)://)?[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}(/[\w\-._~:/?#\[\]@!$&'()*+,;%=]*)?\z}

  friendly_id :slug, use: :slugged

  has_one_attached :logo

  validates :slug, presence: true, uniqueness: true
  validates :name, presence: true
  validates :link, presence: true, format: { with: URL_REGEX, message: I18n.t('activemodel.errors.messages.url_format') }
  validates :logo, attached: true, content_type: ['image/png', 'image/jpg', 'image/jpeg']

  def authorization_definitions
    @authorization_definitions ||= AuthorizationDefinition.all.select do |authorization_definition|
      authorization_definition.provider.id == id.to_s
    end
  end

  def reporters
    users_for_roles(%w[instructor reporter])
  end

  def instructors
    users_for_roles(%w[instructor])
  end

  private

  def users_for_roles(roles)
    User.where(
      "EXISTS (
        SELECT 1
        FROM unnest(roles) AS role
        WHERE role in (?)
      )",
      roles.map { |role| build_user_role_query_param(role) }.flatten,
    )
  end

  def build_user_role_query_param(role)
    authorization_definitions.map do |authorization_definition|
      "#{authorization_definition.id}:#{role}"
    end
  end
end
Key changes:

Inherit from ApplicationRecord (not StaticApplicationRecord)
Add friendly_id configuration (slug-based, like Authorization model)
Add has_one_attached :logo for ActiveStorage
Add validations (name, slug, link format, logo attachment)
Keep existing methods: authorization_definitions, reporters, instructors, users_for_roles, build_user_role_query_param
Remove: attr_accessor, self.backend method
Update authorization_definitions to compare with id.to_s (database id vs static string)

Notes:

URL regex matches pattern from app/models/concerns/authorization_extensions/cadre_juridique.rb
ActiveStorage validations use active_storage_validations gem
Logo content types match existing files (png, jpg, jpeg)

Test: Model tests (see Step 1.4)
Step 1.4: Write Model Tests

File: spec/models/data_provider_spec.rb
Update/add tests:
RSpec.describe DataProvider do
  describe 'validations' do
    subject { build(:data_provider) }

    it { is_expected.to validate_presence_of(:slug) }
    it { is_expected.to validate_presence_of(:name) }
    it { is_expected.to validate_presence_of(:link) }
    it { is_expected.to validate_uniqueness_of(:slug) }

    describe 'link format validation' do
      it 'accepts valid URLs' do
        valid_urls = [
          'https://www.example.gouv.fr',
          'http://example.gouv.fr',
          'https://example.gouv.fr/path/to/page',
        ]

        valid_urls.each do |url|
          subject.link = url
          expect(subject).to be_valid, "Expected #{url} to be valid"
        end
      end

      it 'rejects invalid URLs' do
        invalid_urls = ['not a url', 'ftp://example.com', 'example', '']

        invalid_urls.each do |url|
          subject.link = url
          expect(subject).not_to be_valid, "Expected #{url} to be invalid"
        end
      end
    end

    describe 'logo attachment' do
      it 'validates logo is attached' do
        provider = build(:data_provider)
        provider.logo.purge
        expect(provider).not_to be_valid
        expect(provider.errors[:logo]).to be_present
      end

      it 'validates logo content type' do
        provider = build(:data_provider)
        provider.logo.attach(io: StringIO.new('content'), filename: 'test.txt', content_type: 'text/plain')
        expect(provider).not_to be_valid
      end

      it 'accepts valid image types' do
        %w[image/png image/jpg image/jpeg].each do |content_type|
          provider = build(:data_provider)
          provider.logo.attach(io: StringIO.new('content'), filename: 'test.png', content_type:)
          expect(provider).to be_valid, "Expected #{content_type} to be valid"
        end
      end
    end
  end

  describe 'friendly_id' do
    it 'finds by slug using friendly.find' do
      provider = create(:data_provider, slug: 'test-provider')
      expect(DataProvider.friendly.find('test-provider')).to eq(provider)
    end

    it 'raises error when slug not found' do
      expect { DataProvider.friendly.find('nonexistent') }.to raise_error(ActiveRecord::RecordNotFound)
    end
  end

  describe '#authorization_definitions' do
    it 'returns authorization definitions for this provider' do
      provider = create(:data_provider, slug: 'dgfip')

      definitions = provider.authorization_definitions

      expect(definitions).to be_all { |d| d.provider.id == 'dgfip' }
    end
  end

  describe '#reporters' do
    subject { create(:data_provider, :dgfip).reporters }

    let!(:valid_users) do
      [
        create(:user, :reporter, authorization_request_types: %w[api_impot_particulier api_hermes]),
        create(:user, :instructor, authorization_request_types: %w[api_hermes]),
      ]
    end

    let!(:invalid_users) do
      [
        create(:user, :reporter, authorization_request_types: %w[api_entreprise]),
      ]
    end

    it { is_expected.to match_array(valid_users) }
  end

  describe '#instructors' do
    subject { create(:data_provider, :dgfip).instructors }

    let!(:valid_users) do
      [
        create(:user, :instructor, authorization_request_types: %w[api_impot_particulier api_hermes]),
        create(:user, :instructor, authorization_request_types: %w[api_hermes]),
      ]
    end

    let!(:invalid_users) do
      [
        create(:user, :reporter, authorization_request_types: %w[api_hermes]),
        create(:user, :reporter, authorization_request_types: %w[api_entreprise]),
      ]
    end

    it { is_expected.to match_array(valid_users) }
  end
end
Notes:

Reuse existing test patterns from current spec/models/data_provider_spec.rb
Add validation tests for all new validations
Test friendly_id functionality
Keep existing reporters and instructors tests (should still work)
Use factory with traits for common providers

Test: Run specs
bundle exec rspec spec/models/data_provider_spec.rb
Phase 2: Data Migration

Step 2.1: Create Data Migration

File: db/migrate/YYYYMMDDHHMMSS_migrate_data_providers_from_yaml.rb
FIXME utilise attach! et raise une erreur si le logo n'existe pas
ou n'est pas un png ou jpg
Migration to import YAML data:
class MigrateDataProvidersFromYaml < ActiveRecord::Migration[8.0]
  def up
    providers_data = Rails.application.config_for(:data_providers)

    providers_data.each do |slug, attributes|
      provider = DataProvider.create!(
        slug:,
        name: attributes['name'],
        link: attributes['link']
      )

      logo_filename = attributes['logo']
      logo_path = Rails.root.join('app', 'assets', 'images', 'data_providers', logo_filename)

      if File.exist?(logo_path)
        content_type = case File.extname(logo_filename).downcase
                       when '.png' then 'image/png'
                       when '.jpg', '.jpeg' then 'image/jpeg'
                       else 'application/octet-stream'
                       end

        provider.logo.attach(
          io: File.open(logo_path),
          filename: logo_filename,
          content_type:
        )
      else
        Rails.logger.warn "Logo file not found: #{logo_path}"
      end
    end
  end

  def down
    DataProvider.destroy_all
  end
end
Notes:

Use up/down methods (not change) to allow custom logic
Read YAML using Rails.application.config_for(:data_providers) (same as current code)
Handle different image formats (png, jpg, jpeg)
Add warning logging for missing files
Reversible migration (down method destroys all records)

Test: Run migration in test environment
RAILS_ENV=test bundle exec rails db:migrate
Verify:
RAILS_ENV=test bundle exec rails runner "puts DataProvider.count" # Should be 12
RAILS_ENV=test bundle exec rails runner "puts DataProvider.all.map(&:slug).join(', ')"
Step 2.2: Verify Data Integrity

Manual verification after migration:
RAILS_ENV=test bundle exec rails runner "
  DataProvider.all.each do |p|
    puts '#{p.slug}: #{p.name}, logo: #{p.logo.attached?}, link: #{p.link}'
  end
"
Check:

All 12 providers created
All slugs match YAML keys (dgs, dila, dinum, dgfip, etc.)
All logos attached
All links populated

Phase 3: Update Code References

Step 3.1: Update AuthorizationDefinition

File: app/models/authorization_definition.rb:47
Change:
# Before
provider: DataProvider.find(hash[:provider]),

# After
provider: DataProvider.friendly.find(hash[:provider]),
Notes:

Most critical change - used during AuthorizationDefinition loading
AuthorizationDefinition is still a StaticApplicationRecord loading from YAML
The provider slug in authorization_definitions YAML must match DataProvider slugs

Test: Verify authorization definitions load correctly
RAILS_ENV=test bundle exec rails runner "puts AuthorizationDefinition.all.count"
Step 3.2: Update DGFIP Export Controller

File: app/controllers/dgfip/export_controller.rb:38
Change:
# Before
@data_provider ||= DataProvider.find('dgfip')

# After
@data_provider ||= DataProvider.friendly.find('dgfip')
Test: Verify controller (if specs exist)
Step 3.3: Update Application Helper for Logo Display

File: app/helpers/application_helper.rb:17-20
FIXME le logo est forcément présent
Change:
# Before
def provider_logo_image_tag(authorization_definition, options = {})
  options = options.merge(alt: "Logo du fournisseur de données \" #{authorization_definition.provider.name}\"")
  image_tag("data_providers/#{authorization_definition.provider.logo}", options)
end

# After
def provider_logo_image_tag(authorization_definition, options = {})
  options = options.merge(alt: "Logo du fournisseur de données \" #{authorization_definition.provider.name}\"")

  if authorization_definition.provider.logo.attached?
    image_tag(authorization_definition.provider.logo, options)
  else
    # Fallback if logo not attached (should not happen after migration)
    Rails.logger.warn "Logo not attached for provider: #{authorization_definition.provider.slug}"
    ''
  end
end
Notes:

Use ActiveStorage's image_tag helper (works directly with attachments)
Add safety check for attached logo
Log warning if logo missing (shouldn't happen)

Test: Manual testing in views that use provider logos
Phase 4: Testing and Validation

Step 4.1: Run All RSpec Tests

Command:
bundle exec rspec
Verify:

DataProvider model tests pass
All related model tests pass (AuthorizationDefinition, User)
Controller tests pass (if any for dgfip export)
Helper tests pass (if any)
No regressions in other tests

Fix failures as they occur
Step 4.2: Run Cucumber Features

Command:
bundle exec cucumber
Verify:

All E2E tests pass
Features using DataProvider work correctly
Logo display works in views

Fix failures as they occur
Step 4.3: Manual Testing in Development

Start server:
bin/local_run.sh
Test:


Check DataProvider records exist:
DataProvider.count # Should be 12
DataProvider.all.map(&:slug)


Test friendly_id lookups:
DataProvider.friendly.find('dgfip')
DataProvider.friendly.find('dinum')


Check logos attached:
DataProvider.all.each { |p| puts "#{p.slug}: #{p.logo.attached?}" }


Test relationships:
dgfip = DataProvider.friendly.find('dgfip')
dgfip.authorization_definitions.map(&:id)
dgfip.reporters.count
dgfip.instructors.count


Browse pages that display provider logos


Test DGFIP export functionality (if accessible)


Step 4.4: Run Linters

Commands:
bundle exec rubocop
bundle exec rubocop -A  # Auto-fix issues
Fix any style violations:

Maximum method length: 15 lines
Class length: max 150 lines
Use single quotes for strings
2 spaces indentation

Phase 5: Documentation

Step 5.1: Add Comment to YAML File

File: config/data_providers.yml
Add comment at the top:
---
# NOTE: This file is kept for reference purposes only.
# Data providers are now stored in the database (data_providers table).
# To add new providers, use the Rails console:
#   provider = DataProvider.create!(
#     name: 'Provider Name',
#     link: 'https://provider.gouv.fr'
#   )
#   provider.logo.attach(io: File.open('path/to/logo.png'), filename: 'logo.png', content_type: 'image/png')
#
# This file was used for the initial data migration.
# Migration date: [INSERT DATE]

shared:
  dgs:
    name: Organisation de la direction générale de la santé (DGS)
    logo: ministere_sante.jpg
    link: https://www.sante.gouv.fr/
  # ... rest of file
Notes:

Keep YAML file as backup/reference (per user decision)
Add clear instructions for future additions via console
Document migration date

Step 5.2: Update Technical Documentation (if exists)

Check these files:
ls docs/
Update if DataProvider is mentioned:

Architecture diagrams
Data model documentation
Development guides

Document:

DataProvider is now database-backed
Uses friendly_id for slug-based lookups
Uses ActiveStorage for logo management
How to add new providers via console


Testing Strategy

Following TDD approach per repository guidelines:
Step 1: Models


Create migration
Create factory
Write failing model tests
Implement model changes
Run tests until passing
Run rubocop and fix issues
Refactor if needed

Step 2: Data Migration


Write migration
Test in test environment
Verify data integrity
Test idempotency (run twice, check no duplicates)
Test rollback (down migration)

Step 3: Code Updates


Update each file one at a time
Run related tests after each change
Fix failures
Run rubocop

Step 4: Integration


Run full RSpec suite
Run full Cucumber suite
Manual testing in development
Fix any issues


Rollback Strategy

If migration fails in production:


During deployment: Deployment will fail and rollback automatically (per user decision)


After deployment:

Run down migration: rails db:rollback STEP=2 (2 migrations: data + table)
YAML file still exists as backup
Code will need to be reverted via git


Data recovery:

YAML file remains untouched
Can re-import from YAML if needed
Asset logos remain in app/assets/images/data_providers/


Success Criteria


 DataProvider table created with all required columns and indexes
 Factory created for testing
 All 12 existing providers migrated to database with correct data
 All logos attached via ActiveStorage and display correctly
 All validations working (name, logo, link presence and format)
 friendly_id working for slug-based lookups
 All code references updated to use .friendly.find()
 Application helper updated to use ActiveStorage URLs
 All existing functionality preserved (authorization_definitions, reporters, instructors)
 All RSpec tests passing
 All Cucumber features passing
 Rubocop passing with no violations
 No regressions in application behavior
 YAML file kept with documentation comment
 Manual testing completed successfully


Estimated Implementation Order


Create table migration (30 min)
Create FactoryBot factory (30 min)
Update DataProvider model (1 hour)
Write model tests (1.5 hours)
Run tests and fix issues (1 hour)
Create data migration script (1 hour)
Test data migration in test environment (30 min)
Update AuthorizationDefinition (15 min)
Update DGFIP export controller (15 min)
Update application helper (30 min)
Run all RSpec tests (30 min)
Fix any failing tests (1 hour buffer)
Run Cucumber features (30 min)
Fix any failing features (30 min)
Run rubocop and fix style issues (30 min)
Manual testing (1 hour)
Add YAML comment and documentation (30 min)

Total estimated time: ~12 hours

Risk Assessment

High Risk


Data migration integrity: Losing or corrupting provider data

Mitigation: Test thoroughly in test environment first, keep YAML backup, reversible migration


Breaking AuthorizationDefinition loading: Critical part of the app

Mitigation: Extensive testing, verify all definitions load correctly


Logo display breaking: Views depend on logo display

Mitigation: Test helper thoroughly, add fallback, manual testing


Medium Risk


Missing logo files: Some logos might not exist or have wrong filenames

Mitigation: Verify all files exist before migration, add error handling, warning logs


URL validation too strict: Existing URLs might not match regex

Mitigation: Test regex against all existing URLs from YAML first


ID comparison issues: Database IDs (integers) vs YAML IDs (strings)

Mitigation: Use .to_s in comparisons, test authorization_definitions relationship thoroughly


Low Risk


Performance: Database queries instead of in-memory lookups

Mitigation: Add database indexes on slug, use Rails query cache


Slug conflicts: Unlikely with current data

Mitigation: friendly_id handles this with candidate generation


Notes


This migration follows the same pattern as the Authorization model's use of friendly_id
No controllers or views will be created (backend-only migration)
Future DataProvider management via Rails console only
YAML file kept as backup per user decision
StaticApplicationRecord class remains (other models still use it)
Uses same ActiveStorage service as rest of application
All existing functionality (reporters, instructors, authorization_definitions) preserved
Logo display updated to use ActiveStorage instead of asset pipeline


Open Questions

None - all questions have been answered by the user.
No results found