How to migrate files and images to a media field in Drupal 8

When migrating content to Drupal 8 from another source, it’s common for some fields to be images or files. Here we’ll see how to import them to a media field.

For some time now, the media module has been incorporated into the core and they have been improving it so that now it’s more useful to put files and images in this type of fields, instead of an image or file field.

The media module allows, among other things, reusing content, having a multimedia library and adding the fields you need. It’s almost a standard for the Drupal projects we carry out at Omitsis.

But when it comes to importing, for now it’s more complicated than doing it directly to a file type field. Here we’ll see how to do it.

This post is a continuation of a previous article where we saw how to use the migrate module to import from a csv. It’s recommended to read this article first if you don’t have experience with the migrate module.

There’s a module that allows migrating file entities to media entities, the migrate files to media. It also allows importing them from Drupal 7. In this case we’re not going to use it, we’ll only use the migrate module and the ones we mentioned in the previous post.

First we install these modules with composer:

composer require drupal/migrate_plus drupal/migrate_tools drupal/migrate_source_csv drupal/config_devel

And we enable them:

drush -y en migrate migrate_plus migrate_tools migrate_source_csv config_devel

All these modules are in Drupal 8 and Drupal 9, except config_devel which isn’t yet for version 9, but it seems it will be soon. So everything we’re going to explain will also work for Drupal 9.

We’ll first create a yml to import the images to files. We’ll call this file migrate_plus.migration.migrate_files.yml.

id: migrate_files
label: Migrate Files
source:
  plugin: 'csv'
  # Full path to the file.
  path: 'modules/custom/custom_migrate/data/products.csv'
  # Column delimiter. Comma (,) by default.
  delimiter: ','
  # Field enclosure. Double quotation marks (") by default.
  enclosure: '"'
  # The row to be used as the CSV header (indexed from 0),
  # or null if there is no header row.
  header_offset: 0
  # The column(s) to use as a key. Each column specified will
  # create an index in the migration table and too many columns
  # may throw an index size error.
  ids:
    - code
  # Here we identify the columns of interest in the source file.
  # Each numeric key is the 0-based index of the column.
  # For each column, the key below is the field name assigned to
  # the data on import, to be used in field mappings below.
  # The label value is a user-friendly string for display by the
  # migration UI.
  fields:
    0:
      name: cat
      label: 'Category'
    1:
      name: code
      label: 'Product code'
    2:
      name: title
      label: 'Title'
    3:
      name: image
      label: 'Image'
  constants:
    file_source_uri: 'public://import/source/images'
    file_dest_uri: 'public://imports/dest/images'
process:
  file_source:
    -
      plugin: concat
      delimiter: /
      source:
        - constants/file_source_uri
        - image
    -
      plugin: urlencode

  file_dest:
    -
      plugin: concat
      delimiter: /
      source:
        - constants/file_dest_uri
        - image
    -
      plugin: urlencode

  # We use the image file names as is.
  #
  # Alternatively, if we wish to name them after some other
  # column, we can do it here.
  uri:
    plugin: file_copy
    source:
      - '@file_source'
      - '@file_dest'

  uid:
    plugin: default_value
    default_value: 1

destination:
  plugin: entity:file

migration_dependencies: {}

The first part of this file is almost identical to the one we used to migrate content, since we’ll use the same source, a csv.

Key points to highlight:

• Use of constants to indicate where the images will be and where we’ll put them.
• In process we’ll use two entries to define the source and destination. As you can see it’s simply concatenating the source/destination directory with the image csv field that only has the image name.
• In process in the image uri field we use the previous values with the file_copy plugin. This plugin has more options to define what to do if the file already exists. You can see them in its source code.

The next step is to use these files to create the media entities. We’ll call the file migrate_plus.migration.media_images

id: media_images
label: Media images
source:
  plugin: 'csv'
  # Full path to the file.
  path: 'modules/custom/custom_migrate/data/products.csv'
  # Column delimiter. Comma (,) by default.
  delimiter: ','
  # Field enclosure. Double quotation marks (") by default.
  enclosure: '"'
  # The row to be used as the CSV header (indexed from 0),
  # or null if there is no header row.
  header_offset: 0
  # The column(s) to use as a key. Each column specified will
  # create an index in the migration table and too many columns
  # may throw an index size error.
  ids:
    - code
  # Here we identify the columns of interest in the source file.
  # Each numeric key is the 0-based index of the column.
  # For each column, the key below is the field name assigned to
  # the data on import, to be used in field mappings below.
  # The label value is a user-friendly string for display by the
  # migration UI.
  fields:
    0:
      name: cat
      label: 'Category'
    1:
      name: code
      label: 'Product code'
    2:
      name: title
      label: 'Title'
    3:
      name: image
      label: 'Image'
process:
  # Skip if already existent
  skip:
    -
      plugin: entity_lookup
      value_key: name
      source: image
      bundle_key: bundle
      bundle: image
      entity_type: media
      ignore_case: 1
      access_check: 0
    -
      plugin: skip_on_not_empty
      method: row
      message: 'Skipping already existent node'

  field_media_image/target_id:
    plugin: migration_lookup
    migration: calier_files
    source: code
    no_stub: true
  uid:
    plugin: default_value
    default_value: 1

destination:
  plugin: entity:media
  default_bundle: image

migration_dependencies:
  optional:
    - migrate_files

Key points:

• The initial part is the same
• The skip is to not create a media for each entry in the csv if the file already exists, since what we want with the media is to reuse the images.
• For this we’ll use the custom skip_on_not_empty plugin that I copied from here.
• What the skip does is use the entity_lookup plugin to search for a media entity whose name is the file name. If it finds it with the skip_on_not_empty plugin we’ll get it not to import it and thus avoid duplicates.
• Then with the migration_lookup we’ll indicate the fid of the file we imported previously.
• It only remains to indicate a destination which will be an entity_media. The bundle could be put in the process if it were changing for each row but in this case they are always images, so we can define a default_bundle directly.
• Finally we indicate that this migration depends on the previous one. It’s set as “optional” because with required it gave problems. There’s an issue about this that is being resolved.

And now only the content import itself remains, referencing the created media entities.

id: products
label: Import products

source:
  plugin: 'csv'
  # Full path to the file.
  path: 'modules/custom/custom_migration/data/products.csv'
  # Column delimiter. Comma (,) by default.
  delimiter: ','
  # Field enclosure. Double quotation marks (") by default.
  enclosure: '"'
  # The row to be used as the CSV header (indexed from 0),
  # or null if there is no header row.
  header_offset: 0
  # The column(s) to use as a key. Each column specified will
  # create an index in the migration table and too many columns
  # may throw an index size error.
  ids:
    - code
  # Here we identify the columns of interest in the source file.
  # Each numeric key is the 0-based index of the column.
  # For each column, the key below is the field name assigned to
  # the data on import, to be used in field mappings below.
  # The label value is a user-friendly string for display by the
  # migration UI.
  fields:
    0:
      name: cat
      label: 'Category'
    1:
      name: code
      label: 'Product code'
    2:
      name: title
      label: 'Title'
    3:
      name: image
      label: 'Image'

process:
  field_cat:
    plugin: entity_lookup
    source: division
    value_key: name
    bundle_key: vid
    bundle: cat_producto
    entity_type: taxonomy_term
    ignore_case: true
  field_producto_codigo: code
  title:
    -
      source: title
      plugin: callback
      callable: mb_strtolower
    -
      plugin: callback
      callable: ucfirst
  type:
    plugin: default_value
    default_value: producto
  field_imagen/target_id:
    plugin: entity_lookup
    value_key: name
    source: image
    bundle_key: bundle
    bundle: image
    entity_type: media
    ignore_case: 1
    access_check: 0

destination:
  plugin: entity:node

migration_dependencies:
  optional:
    - migrate_files
    - media_images

Key points:

• To reference the media entities we use the entity_lookup plugin that searches by the image name.
• Everything else we already explained in the previous post.

We indicate these files in the module yml:

config_devel:
  install:
    - migrate_plus.migration.products
    - migrate_plus.migration.migrate_files    
    - migrate_plus.migration.media_images

And now only a few commands remain:

First tell Drupal to take into account the created import ymls.

drush cdi custom_migrate

And then the imports of each of the files:

drush mim migrate_files

drush mim media_images

drush mim products

And with all this we would have ready the import of images to media entities and then created the nodes that contain these media fields.