Our growing business drew the attention of international customers, allowing us to enter new markets. The internationalization of our application was one of the primary requirements in order to perform in this new environment.
The ZEND framework is used for the backend. It was obvious to use .po-files for the translation-mechanism. It wasn’t too difficult to offer our software with only one additional language (english), so it was possible to manage the .po-files manually. The well-known POEdit was the tool of our choice. The developers wrote their code, commited and pushed it into our GIT repository. We then defined a manual process to update the .po-files. This allowed us to either translate them in-house or send them to a translation office.
In the end, as demand for other languages rose, lacking the linguistic expertise in our team to deal with additional languages, led to this process becoming ineffective quickly. Having to handle this amount of translations became a pain-staking process with our limited resources, which inevitably led to an increase in our error-rate. It was time to think it over and define a better process.
The idea was to create a process in the following way:
Every developer codes in the common way we did before. Any apparent string in our code is written with the ZEND translation mechanism.
We didn’t want our developers to think about translations anymore, because their main focus should be on the business logic. The .po-files should automatically be updated when the code is committed and merged into the repo. We want to enable our translation office to use a visual interface for translatiosn adn we would like to synchronize everything without any manual effort. We could define a process like this:
1. Developers code with translation marks and our git-process should remain untouched.
2. The creation of translation files should be fully automatic.
3. We need a translation tool with common features, such as a translations db and online lookup.
4. The exchange of translations which need translating and those already translated should also be automatic.
We defined the following process:
1. The developer implements the code as usual. The code is merged into the master branch after revision.
2. The .po-files are automatically refreshed and checked into the master branch, when the new code is merged into the master branch.
3. A WebLate-server is the tool we want to use to empower translation offices to work with us. WebLate offers all features for professional translation work and can handle git-repositories.
4. The WebLate server must receive the new .po-files automatically.
5. The new .po-files must be checked in automatically after the translators finished their work.
We transformed this process into a real workflow as described in the following:
We use pipelines with jobs in order to automate processes with GITLab. It is therefore necessary to install a runner to execute the jobs. There is quite a good documentation on GITLab you can find here: https://docs.gitlab.com/runner/install/
The essentials for our GITLab installation on AWS Ressources:
The runner must be configured in the settings of your GITLab-project. We disabled shared runners and activated it for our project. You have to enable the runner for every single project.
Prepare the project with job description:
The runner can execute pipelines containing jobs. In order to achieve this, we first created a GITLab-user called „SDGitlabRunner“ (meep meep, nobody is faster) and then assigned this user to every project we needed. The easiest way is to make him a master, but you can also make a more complex process and only give him developer rights on a special branch.
You must create an SSH-Keypair for this user before you can assign the GITLab-Runner-User. This significant point is the only setting which leaves me a little bit uncomfortable. It is necessary to assign the public key as usual to the GITLab-User. For the script in the next chapter, on the other hand, it is necessary to set a private key as a „secret variable“ with the name „SSH_PRIVATE_KEY“ in every project. Not even Google could find a better solution. I don’t necessarily like it this way, but we need it to push our updated .po-files back to the repo.
The runner with all the jobs is controlled with one file. So we created a .gitlab-ci.yml-file in the root directory of every project which needs translations. Don’t forget to add the runner for every desired project.
The yml-file we created looks finally like the following:
So whats going on here? Let’s go though it step by step.
The before_script is executed before all other jobs. We create an .ssh-private key and catch it from the environment variable. We also needed to create a known_host-file executing
ssh-keygen -R [hostname] or use StrictHostKeyChecking no, but the first way is the better way. In the before_script we also set the right GIT-remotes, the username and the e-mail-address and switched to push.default simple.
The next job is to check out the current code. We have to keep the files between the jobs, so we active the cache. We use the hard way for checkout to avoid any merge-conflict.
Now we grep through the source-code-directories and put every file path we want to consider in a file called potfile.txt.
the next job is to grep through the extracted files and find translations („this->translate“) in the code. A lot of times, xgettext is not installed on the GITLab-Server, especially if you use default AMIs from AWS. Just install it with:
The result is written in the temporary.pot-file.
Now we conduct a merge of the existing .po-Files and the new translations we found in the source code. Just to be sure we check if the target-.po-file exists. If you have more languages, just add them.
Last step: checking the new .po-files and push them back to the repository. First we remove unneeded files. We count the number of files to push to avoid „empty“ pushes to the repo wich will cause failed job states for the GITLab pipeline.
Now we can automatically update .po-files if there are changes in the code. The developer doesn’t have to take care of it, he just writes his code.
WebLate and done
The next step is to inform our WebLate-Server about the changes. That’s pretty easy – we just add a web hook from WebLate to every project in GITLab. Go to „settings->integrations“ in the GITLab-project and configure a Webhook – that’s it!
Way back from WebLate to GITLab
The last step in order to accomplish this process is to enable WebLate to push changes back to the repository. We do it the same way again – we create a user in GitLab called „WebLateUser“ and attach this user to the desired projects. We need an SSH-key again that we give to this user in GITLab and configure this on the WebLate-side in the admin-panel. You can read more here:
For mixed views composed with our ZEND Framework we wrote a little script „translation collector“, triggered bevor the find-file-stage. We execute this to extract translations into a .phtml-file bevor we run the „find“.
Dipl.-Ing. (BA) Ronny Rohland
Co-Founder & Lead Developer SimplyDelivery
If you have any question don’t hesitate to contact us.