Back to all posts

Write API for Translating Content

Posted by Fernando Doglio on July 11, 2019

Internationalization is a big topic on any context that involves content. If you’re writing for your company’s website and want the content to reach markets in several countries, you’ll need internationalization. If you’re working on a mobile app that deals with money, you’ll need internationalization. Anything you want to do that reaches or affects users in different countries will, most likely, require internationalization.

In this article, I want to cover the specific use case in which you’ll need several copies of your content instead of dynamically changing parts of it. Particularly in the context of ButterCMS and pages with particular locales.

You’ll learn how to use their new Write API to simplify your life when it comes to localizing your content.

So, let’s get to it, shall we?

The pain of localization

But first, let’s cover the dark side of localizing your content. Localization is the act of customizing your content to the specific locale (geographic location) of the person interacting with your content. In its simplest form, that means translating text into Spanish for a reader from Latin America, and translating it again into English when the reader is located in the United States.

A more complex scenario would involve dynamically changing the currency in order to provide correct prices (based on up-to-date exchange rates) for your e-commerce site, based on the locale of your customers.

In both cases, extra work is required to provide the service of translating content. This may entail either writing several versions of your content in different languages (and then maintaining them over time), or adding the extra code required to make the currency exchange work as expected by either maintaining an active exchange table or adding the required code to pull it from a hopefully, reliably up-to-date 3rd party API . Localization done right can sometimes be the stuff of nightmares for your team. At the same time, it can also be a huge benefit for your website users, making all the hustle worthwhile. This is why we all try to do it at some level.

The key is to do it right and try to help both your users and your team in the process.

Localization with ButterCMS

Since we’re covering ButterCMS’ newest API, the Write API, let’s first discuss how to add localization to your site, following the normal path (A.K.A by hand).

In order to enable localization within Butter, you’ll have to go to your profile (click on your name on the top left corner of the page) and select the “Settings” option.

Translating content buttercms api

After that, simply select the “Localization” tab, and start adding how many locales you’ll need for your pages for translation.

translating content

Once that is done, creating several versions of your pages, one for each locale you created is relatively easy.  Simply select the locale from the list of available options on the list of pages:

translating content

That’s it! You’re officially playing the localization game with your own pages, and it works great.  You don’t have to worry about it. You can even request the correct locale based on your user preferences from Butter’s API and it’ll return it for you, without having to do anything special. 

That being said, I’m sure you can see how if we were to scale the above scenario at any significant level, either vertically (adding a lot of new pages) or horizontally (adding a lot of new locales for each page) then that amazing feature you’ve added to your site, will become a large task. 

But fear not! This is one way Butter’s new Write API can help you manage your content. We’re going to be looking at a way to automate your workflow for translating content in the hopes of minimizing your involvement in the localization process of your own content. Sounds exciting? That’s because it is! 

Solving the localization headache

My intent with the rest of this article will be to show you how to set up an automated workflow, using ButterCMS and a bit of Node.js code, to automatically create or update a single page and all of your locales tied to it.

Here is the main idea:

Write API Translation Workflow

The way to read the above diagram would be:

  1. The author creates a new page using ButterCMS, specifying one particular locale (in this example “en” for English).
  2. A webhook is triggered once the new page is saved. This hook executes a script sending it the page data.
  3. The script, in turn, will use the Google Translate API in order to get a Spanish translation of the content of the page.
  4. Once received, the script will then save the translated content as a new version of the original page, using a different locale.

Simple enough, isn’t it? Let’s now take a look at the actual implementation.

A Use Case for Content Translation

For the purposes of this article, I’m going to be implementing this website below as an example for reference.

translating content buttercms
The above screenshot shows a typical commercial website, showing a set of products with a description. The way I’ve set this up in Butter is by creating a single product page with three different fields. Each of these fields represents the name and description of each category. 

Here is the page in ButterCMS:

translating content

Once your page is created, we can set up a content translation process that gets triggered once we make an update to the page to perform the translation of the English content for each of our three WYSIWYG fields into Spanish without any direct interaction.

Planning the Implementation

The implementation of this solution for translating content is just as straightforward as the architecture diagram shown above. Let’s quickly review the APIs we’re about to use so we’re all on the same page. 

Google Translate

For the translation service, obviously, we’ll be using this particular API. In order to set it up, you need to:

  1. Go to GoogleCloud’s Web Console and sign-in.
  2. Once there, search for the Translate API and enable it. Do note, that in order to do that, you’ll have to enter payment information (by default you have up to 500k characters a month for free to test the API, mind you, but you’ll still have to enable payments regardless).
  3. Finally, you’ll need to download the JSON file with your account’s credentials. You can see how to do that here (pay special attention to setting up the environment variable called “GOOGLE_APPLICATION_CREDENTIALS” otherwise nothing will work for you).

 Once the above steps are done, you’re all set to start interacting with that API, and since we’re going to be using Node.js for our coding needs, you can simply use Google’s official npm module.

Getting a translation from the API using this library is as simple as doing the following: 

async function quickstart(

  projectId = 'YOUR_PROJECT_ID' // Your GCP Project Id

) {

  // Imports the Google Cloud client library

  const {Translate} = require('@google-cloud/translate');

 

  // Instantiates a client

  const translate = new Translate({projectId});

 

  // The text to translate

  const text = 'Hello, world!';

 

  // The target language

  const target = 'ru';

 

  // Translates some text into Russian

  const [translation] = await translate.translate(text, target);

  console.log(`Text: ${text}`);

  console.log(`Translation: ${translation}`);

}

Another great benefit is that this API ignores markup code in the text you send it. So, as I’ll show you in a bit, we can simply send a bunch of HTML code and we’ll get the same tags with their content translated (awesome!)

ButterCMS APIs

Even though the process will start as part of the Webhook being triggered, we’ll still need to interact with both of Butter’s APIs, Read and Write. Together they will allow us to create the new pages directly from our code.

Just like with Google’s API, we do have an npm module for Butter as well, called buttercms. Do note however, that this module only takes care of the interaction with the Read API, so we’ll have to manually tackle the Write API ourselves (not to worry though, it’s just a single HTTP request).

For the purposes of this article, we only need to perform a single API call into the Read API, in order to retrieve a single page, the page that is being updated. Because we’re only looking for the English version of the page (remember from above, where we defined the flow?), we’ll have to add an extra parameter to call.

Here is how:

butter.page.retrieve(“page type”, “your page slug”, {locale: 'en'}).then( (page) => {

//... your code here

})

Notice the third parameter, {locale: 'en'},that’s all we need to let their API know we’re looking for one particular version of the page. If you want to know more (such as getting a paginated list of pages, or creating a new page dynamically), you can read the official online docs for their API here. 

Setting up the Webhook trigger

The final piece of the content translation puzzle, involves setting up the trigger for the entire flow. Lucky for us, ButterCMS already has support for triggering webhooks when a particular action happens. They support actions such as: creating a new blog post, publishing a blog post, creating a new page, deleting an existing page, and so on. I invite you to see the full list of events in their online docs here. For any and all of the events they support, once they happen, a POST request will be triggered to a particular URL that you have set up, with a very specific payload describing the event that just happened (through the use of an event code) and the element that got affected (either a page, a blog post or a collection). That’s all you need really, although as you’ll see in the actual code for the script, you’ll need to have it published somewhere where it can be accessed by Butter. In other words, it’s not just a script you can upload and forget about. You’ll need a dedicated URL for it, which means it’ll have to be hosted somewhere (as a suggestion in case you don’t want to go through the struggle of maintaining a server somewhere, you can try a service such as https://zeit.co/ which provides a serverless like infrastructure for you to host your scripts).

 

That being said, setting the hook is trivially simple, just follow these steps:

  1. Go to the settings section (like before):undefined
  2. Once inside, click on the Webhooks tab:
    undefined
  3. And finally, add the details for your hook like below:undefined

For our use case, you can ignore the Header fields and just setup the URL and the event.

The Implementation

Now that you have the hook and you understand what we’re trying to do, here is the relevant code for the script:

// The target language

const TARGET_LANG = 'es';

 

app.post('/webhook1', async function (req, res) {

 

    let pageSlug = req.body.data.id

    let pageType = req.body.data.page_type

 

    //get the original English version of the page

    butter.page.retrieve(pageType, pageSlug, {locale: 'en'}).then( async (page) => {

    let fields = Object.keys(page.data.data.fields)

 

    //Setup the structure and default values for the translated payload

    let tPage = {

    fields: {

    "es": page.data.data.fields

    },

    status: "draft", //we're going with a draft for the time being, just to make sure it works

    "page-type": page.data.data.page_type

    }

 

    let translations = []

    for(let i = 0; i < fields.length; i++) { //perform the actual translation

    let v = await translate(page.data.data.fields[fields[i]], TARGET_LANG)

    tPage.fields["es"][fields[i]] = v //update the default values to turn them into spanish

    translations.push(v)

    }

    let patchUrl = 'https://api.buttercms.com/v2/pages/*/' + page.data.data.slug + "/"

    request.patch({ //perform the page Patch

    url: patchUrl,

    headers: {

    "Authorization": "Token " + WRITE_TOKEN

    },

    json: true,

    body: tPage,

 

    }, (err, resp) => { //We're done!

    if(err) {

    console.log("There was an error: ", err)

    return res.json(err)

    }

    console.log("Done, patch finished")

    console.log(resp)

    res.json(resp)

    })

    }).catch(err => {

    console.log("There was an error: ", err)

    res.json(err)

    })

})

Note that if you want to look at the full source code for the script, you can do so here. But that being said, the relevant logic is in the route described above.  We’re basically defining an Express.js web server with a single endpoint, a POST to /wehbook1, and that’s the URL you’ll have to set up on the webhook panel.

Once this endpoint is reached it performs the following steps:

  1. Get the English (original) version of the page.
  2. Creates the placeholder structure for the translated version (which will be the payload for our request down the line). This structure is already populated with default values, including the draft status, as well as all fields with their original values.
  3. For each field, the translation is done, just like I mentioned before, and each response is then updated on the translated structure.
  4. Finally, a PATCH request is issued to ButtterCMS’s Write API. We’re basically adding a whole new set of fields (or replacing existing ones) for the Spanish locale. 

That is pretty much it, once Butter’s API is reached, it’ll return a simple 200 OK with a body like the following: 

{ status: 'pending' }

This is normal, since updates are done asynchronously on their side, so it takes a few seconds for everything to catch up.

Back to our Translation Use Case

Remember the fake Freeze website with the ice cream? Remember I told you I showed you how I had it set up in Butter? Here is the code I use to load the content from Butter into my page:

function getPageContent() {

      var butter = Butter(‘my-butter-key’);

 

      let defaultLocale = 'en'

      let locale = defaultLocale

 

      if(window.location.search.indexOf("locale") != -1) {

      locale = window.location.search.split("=")[1]

      }

 

      butter

       .page

       .retrieve('product_pages', 

                 "product-page",  

 {locale: locale}

    ).then(function(response) {

      document.getElementById("berry").innerHTML = response.data.data.fields.berry

      document.getElementById("classics").innerHTML = response.data.data.fields.classics

      document.getElementById("fruits").innerHTML = response.data.data.fields.fruits

    })

    }

It’s very straightforward vanilla JavaScript. I’m not using Angular or React or any of the fancy new frameworks out there. There is no need for any extra, but you should be able to extrapolate the code you’ll need from this and fill in the gaps where needed.

Of course, this is a very specific case, but here is what it does:

  1. This function gets called on the “onload” event of the Body element.
  2. Once it’s called, it checks the URL to see if you’ve specified the locale parameter.
    1. If you haven’t, it’ll use the default value (‘en’)
    2. But if you have, it’ll use whatever you send instead
  3. It’ll call the retrieve method and get the content of our page (by specifying its type and slug, as usual) using the set locale value.
  4. With the response, our function will populate the content of three particular elements in the HTML code. 

Check out the following image, showing side-by-side comparison of both versions of the site, with the only change being the added parameter to the URL:

undefined

Final thoughts

That is it for this article! I showed you how to use Butter’s new Write API and their Webhooks to set up a very simple, yet super powerful auto-translation workflow, allowing you to localize your content pages for translation on all the languages you’d like with very minimum effort. 

All you have to do is:

  1. Grab my code, tweak it to your needs and publish it wherever you can.
  2. Setup your own accounts at ButterCMS and Google Cloud
  3. Configure the Webhook 

And start enjoying your free time!

Also note that you could potentially improve the flow presented here, by adding further steps into the script. You could add calls to other APIs to perform grammar checks, for example, the peeps at ProWritingAid have a very interesting combo you can use to check your writing. You could add a call to their API as well in order to get further insights into what you’re planning on publishing and, you could even send an email to yourself, notifying you of the results.

Thanks for reading, please do let us know in the comments if this tutorial was useful and if you’re having any issues (or success!) with this method.

See you on the next one!

Related Articles