Struggling with migrations using turndown

I’m trying to import a bunch of articles from a site, all of which have HTML formatting.

I’ve been successful at importing the .csv into Storyblok, but of course the ‘long_text’ field content is now screwy as it’s a RichText field in StoryBlok.

Using StoryBlok CLI, I’ve been trying to use the migration tool to migrate the field from HTML to Richtext using turndown, but I’m having no luck.

In the CLI, I’ve successfully generated the migration file, and have set it up by combining the documentation on the StoryBlok github and the Turndown github. My migration file looks like this:

var richtextConverter = require('/opt/homebrew/lib/node_modules/storyblok-markdown-richtext')
var TurndownService = require('/opt/homebrew/lib/node_modules/turndown')

var TurndownService = new TurndownService()

module.exports = function (block) {
  block.intro = richtextConverter.markdownToRichtext(TurndownService.turndown(block.intro))
}

The result in the CLI is:

- Processing the migration ./migrations/change_Post_long_text.js

X An error ocurred when run the migration file: The migration to combination change_Post_long_text.js doesn't exists
me@mymachine migrations % 

I followed this readme:

Combined with this: GitHub - storyblok/storyblok-markdown-richtext: A html/markdown to Storyblok richtext converter
And this: GitHub - mixmark-io/turndown: 🛏 An HTML to Markdown converter written in JavaScript

Help would be greatly appreciated! :slight_smile:

I’ve managed to track this down a little further.

I was able to create a script that used turndown to convert the HTML fields in my CSV to Markdown. That works perfectly. Here’s the script if anyone else if having trouble with this:

import fs from 'fs'
import csv from 'fast-csv'

import TurndownService from 'turndown'

const turndownService = new TurndownService()

function convertHTML(htmlIngest){
  return turndownService.turndown(htmlIngest);
}


const inputFile = 'market-news-input.csv';
const outputFile = 'market-news-output.csv';

(async function () {

  const writeStream = fs.createWriteStream(outputFile);

  const parse = csv.parse(
    { 
      ignoreEmpty: true,
      discardUnmappedColumns: true,
      headers: ['title', 'path', 'author', 'created_at', 'intro', 'videoID', 'long_text', 'image'],
    });

  const transform = csv.format({ headers: true })
    .transform((row) => (
      {
        slug: row.path,
        name: row.title,
        created_at: row.created_at,
        published_at: row.created_at,
        first_published_at: row.created_at,
        title: row.title,
        intro: convertHTML(row.intro),
        image: row.img_thumb,
        videoID: row.videoID,
        long_text: convertHTML(row.long_text)
      }
    ));

  const stream = fs.createReadStream(inputFile)
    .pipe(parse)
    .pipe(transform)
    .pipe(writeStream);
})();

I’m now left with two problems:

  1. I can’t for the life of me get storyblok-markdown-richtext working

I decided to try this as a standalone script, rather than combine it with migrate, but couldn’t get it to play ball. I’ve been using the GitHub examples, but get the following:

import { markdownToRichtext } from 'storyblok-markdown-richtext'
         ^^^^^^^^^^^^^^^^^^
SyntaxError: Named export 'markdownToRichtext' not found. The requested module 'storyblok-markdown-richtext' is a CommonJS module, which may not support all module.exports as named exports.
CommonJS modules can always be imported via the default export, for example using:

import pkg from 'storyblok-markdown-richtext';
const { markdownToRichtext } = pkg;

When I try the suggested fix, the fields that should be converted to RT contain [object, object] instead.

  1. The StoryBlok CLI migrate is also no longer generating migration files for me:
X An error occurred when loading the components from space: Unauthorized
X An error ocurred when generate the migration file: Request failed with status code 401

Despite still being logged into the Storyblok CLI and website.