Installation guide

Important notice

You don’t need to install MateCat in order to use it: just visit the page http://www.matecat.com to translate with MateCat.
This guide is meant for users who want to install and administer the open source version on their own machines.

Table of content

Overview

MateCat is a client-server web application. Most of the interaction runs in the user browser (client), while all calculation along with user data permanently resides on the server side (server).
To run MateCat as a end-user connected to a remote server you only need:

  1. Google Chrome browser
  2. A machine with a few GB of RAM
  3. Internet connection

To run MateCat server on your personal machine (keeping all data on your machine and bearing all the load), you also need administration privileges to install Apache, PHP and MySQL. Running MateCat locally on your machine does not require an Internet connection.
Go to top

Hardware requirements

If you just want to test MateCat on a few million words you will need 1 server with:

  • Storage: 50GB
  • CPU: 2 core x 2.5 Ghz
  • RAM: 4GB

To manage 50M words per year of translation capacity, you will need 1 server with:

  • Storage: 1TB
  • CPU: 6 core x 2.5 Ghz
  • RAM: 32GB

If you need to manage up to 200M words per year, you can do it using 4TB of storage, 12 CPU core, and 64GB of RAM and placing the MySQL database on a separate machine (easy to do with the MateCat config file). For handling more than 200M words per year you will need to add a load balancer in front of the web servers and databases. Databases will need a master-master replica schema. This will require an experienced LAMP and scalability engineer.
Go to top

Advanced Manual Setup

OS

This guide will assume you are on Ubuntu 18 Linux distribution or higher.
You will be guided in the installation of all the necessary dependencies,
if there is any package that you have already installed on your server you can skip the step.
Go to top

Install Web Server: Apache2

sudo su
apt-get update
apt-get install apache2

Go to top

Install Redis

Different from ActiveMQ, Redis will automatically start on boot.

apt-get install redis-server

Make redis listen on all ports

sed -i 's/bind 127.0.0.1/bind 0.0.0.0/g' /etc/redis/redis.conf

Done! You can launch it by typing

service redis-server restart

If you cannot start Redis on the default port you can change it:

redis-server –port 6400

Go to top

Install MySQL

This command installs mysql v5.7.36:

apt-get install mysql-server mysql-client

Go to top

Install PHP5.6 additional packages

Use this command if add-apt-repository is not installed:

apt-get install software-properties-common

Then:

add-apt-repository ppa:ondrej/php
add-apt-repository ppa:ondrej/apache2
apt-get update
apt-get -y --fix-missing install libapache2-mod-php5.6 \
 php5.6 php5.6-json php5.6-curl php5.6-mysql php5.6-xml php5.6-mbstring php5.6-dev php5.6-mcrypt php5.6-redis \
 php5.6-zip

Go to top

Install other packages

apt install zip unzip screen

Go to top

Enable some modules

Apache web server needs to be fine tuned for MateCat to work properly. Enter these commands to enable required Apache submodules

a2enmod rewrite filter deflate headers expires proxy_http ssl php5.6
phpenmod mcrypt
service apache2 restart

Go to top

Configure MySql

Save your current configuration

cp /etc/mysql/mysql.conf.d/mysqld.cnf /etc/mysql/mysql.conf.d/mysqld.cnf.bkp

Paste the new configuration

vi /etc/mysql/mysql.conf.d/mysqld.cnf
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
 
[mysqld_safe]
socket = /var/run/mysqld/mysqld.sock
nice = 0
 
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
skip-name-resolve

init_connect='SET NAMES utf8'
skip-character-set-client-handshake
character-set-server=utf8

bind-address = 0.0.0.0

key_buffer_size = 32M
max_allowed_packet=64M
thread_stack = 192K
thread_cache_size = 50
myisam-recover-options = BACKUP
max_connections = 500
open_files_limit = 65535
table_definition_cache = 1024
table_open_cache = 2048
query_cache_limit = 1M
query_cache_size = 16M
slow-query-log = 1
slow-query-log-file = /var/log/mysql/mysql-slow.log
long_query_time = 1000000
log-queries-not-using-indexes=1
log_error = /var/log/mysql/error.log

expire_logs_days = 10
max_binlog_size = 100M

gtid_mode = on
enforce_gtid_consistency = true
sql-mode="NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
explicit_defaults_for_timestamp=true

innodb_use_native_aio = 0
innodb-flush-method = O_DIRECT
innodb_file_per_table = 1
innodb-buffer-pool-size = 1G
innodb_read_io_threads=64
innodb-log-files-in-group = 2
innodb_write_io_threads=64
innodb-flush-log-at-trx-commit = 2
innodb_log_file_size=256M

[mysqldump]
quick
quote-names
max_allowed_packet = 32M
 
[mysql]
 
[isamchk]
key_buffer = 16M
!includedir /etc/mysql/conf.d/

Restart mysql service

service mysql restart

Go to top

Install Active MQ

ActiveMQ requires an installed version of Java. If you don’t have java installed just type:

apt-get install openjdk-11-jre -y

Download ActiveMQ 5.15

wget http://archive.apache.org/dist/activemq/5.15.8/apache-activemq-5.15.8-bin.tar.gz

Extract the tarball

tar -xvzf apache-activemq-5.15.8-bin.tar.gz

Move it to the /opt folder

mv apache-activemq-5.15.8 /opt/activemq

 

Create an “activemq” user and allow it to execute bash scripts

adduser -system activemq

Change ownership of activemq folders

chown -R activemq: /opt/activemq/

Sym-link the init script provided by Active MQ to /etc/init.d/activemq

ln -sf /opt/activemq/bin/activemq /etc/init.d/

Create a default configuration file with ActiveMQ, change ownership and group for this file and make it accessible only by root

/etc/init.d/activemq create /etc/default/activemq
chown root:nogroup /etc/default/activemq
chmod 600 /etc/default/activemq

Enable ActiveMQ connector for the web interface

sed -i 's/Context createConnector='false'/Context createConnector='true'/g' /etc/default/activemq/conf/activemq.xml

Sym-link ActiveMQ into /bin/bash so that it can be easily called into the terminal

ln -s /etc/init.d/activemq /usr/bin/activemq

Done! You can now start ActiveMQ by typing

activemq start

 

Go to top

Install Node 14

Node.js is necessary to have a real-time chat within Matecat. Install it with:

curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.0/install.sh | bash
vi ~/.zshrc

Edit this file:

export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" 
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"
source ~/.nvm/nvm.sh

Finally, install node 14:

nvm install 14.15.5




Go to top




Tune PHP configuration




echo "short_open_tag = On" >> /etc/php/5.6/cli/php.ini






echo "memory_limit = 1024M" >> /etc/php/5.6/cli/php.ini






service apache2 restart




Go to top




Install MateCat


This section assumes you’ll install MateCat under the home of user matecat (/home/matecat).

In order to create that user, use the following:




adduser --disabled-password --gecos "" matecat






apt-get install git






su - matecat




Then clone the repository in /home/matecat/matecat (feel free to change folder name):




git clone https://github.com/matecat/MateCat.git matecat






exit






cd /home/matecat/matecat/INSTALL






mysql -u root -p < matecat.sql






mysql -u root




Feel free to specify the privileges you want, with the next two instructions you will be able to access only from localhost




delete from mysql.user where user = 'matecat' and host = '%';






grant all privileges on matecat.* to 'matecat'@'localhost' identified by 'matecat01';


FLUSH PRIVILEGES;


exit




Go to top




Create the virtualhost


Navigate to the directory




cd /etc/apache2/sites-available




Remove other virtualhosts:




rm *




Paste the configuration in the default virtualhost:




vi default-vhost.conf






<VirtualHost *:80>
 RewriteEngine On
 RewriteCond %{HTTPS} !=on
 RewriteCond %{SERVER_PORT} 80
 RewriteCond %{HTTP_HOST} ^(.*)$
 RewriteRule ^/(.*)$ https://%1/$1 [R]
</VirtualHost>





Paste the configuration in the Matecat virtualhost:




vi 443-matecat.conf






<VirtualHost *:443>

 ServerAdmin webmaster@localhost
 ServerName your_domain.com

 DocumentRoot /home/matecat/matecat
 DirectoryIndex index.php index.php3 index.html index.htm index.shtml
 <Directory />
 Options FollowSymLinks
 AllowOverride None
 </Directory>

 SSLEngine On
 SSLProtocol all -SSLv3 -SSLv2

 SSLProxyEngine On

 SSLCertificateFile /etc/apache2/ssl-cert/my_matecat.crt
 SSLCertificateKeyFile /etc/apache2/ssl-cert/my_matecat.key


 <Directory /home/matecat/matecat/ >

 Options All
 AllowOverride All
 Require all granted

 ExpiresActive On
 ExpiresByType text/html "access plus 1 minute"
 ExpiresByType text/css "access plus 1 minute"
 ExpiresByType text/javascript "access plus 1 minute"
 ExpiresByType image/gif "access plus 1 week"
 ExpiresByType image/jpeg "access plus 1 week"
 ExpiresByType image/jpg "access plus 1 week"
 ExpiresByType image/png "access plus 1 week"
 ExpiresByType image/vnd.microsoft.icon "access plus 1 week"
 ExpiresByType image/ico "access plus 1 week"
 ExpiresByType application/x-shockwave-flash "access plus 1 week"

 </Directory>

 php_flag register_globals off
 php_flag magic_quotes_gpc off
 ErrorLog ${APACHE_LOG_DIR}/matecat/error.log
 CustomLog ${APACHE_LOG_DIR}/matecat/access.log combined
 ServerSignature Off

 <Location /sse/ >
 ProxyPass http://localhost:7788/
 ProxyPassReverse http://localhost:7788/
 </Location>

</VirtualHost>

<VirtualHost *:443>

 ServerAdmin webmaster@localhost
 ServerName 0.ajax.your_domain.com
 ServerAlias 1.ajax.your_domain.com
 ServerAlias 2.ajax.your_domain.com
 ServerAlias 3.ajax.your_domain.com

 DocumentRoot /home/matecat/matecat
 DirectoryIndex index.php index.php3 index.html index.htm index.shtml
 <Directory />
 Options FollowSymLinks
 AllowOverride None
 </Directory>

 SSLEngine On
 SSLProtocol all -SSLv3 -SSLv2

 SSLProxyEngine On

 SSLCertificateFile /etc/apache2/ssl-cert/my_matecat.crt
 SSLCertificateKeyFile /etc/apache2/ssl-cert/my_matecat.key

 <Directory /home/matecat/matecat/>

 Options All
 AllowOverride All
 Order allow,deny
 allow from all

 ExpiresActive On
 ExpiresByType text/html "access plus 1 minute"
 ExpiresByType text/css "access plus 1 minute"
 ExpiresByType text/javascript "access plus 1 minute"
 ExpiresByType image/gif "access plus 1 week"
 ExpiresByType image/jpeg "access plus 1 week"
 ExpiresByType image/jpg "access plus 1 week"
 ExpiresByType image/png "access plus 1 week"
 ExpiresByType image/vnd.microsoft.icon "access plus 1 week"
 ExpiresByType image/ico "access plus 1 week"
 ExpiresByType application/x-shockwave-flash "access plus 1 week"

 </Directory>

 php_flag register_globals off
 php_flag magic_quotes_gpc off
 ErrorLog ${APACHE_LOG_DIR}/matecat/error.log
 CustomLog ${APACHE_LOG_DIR}/matecat/access.log combined
 ServerSignature Off

 <Location /sse/ >
 ProxyPass http://localhost:7788/
 ProxyPassReverse http://localhost:7788/
 </Location>

</VirtualHost>




Please pay attention to your domain name. For example, if you want to install Matecat on your localhost replace your_domain.com with localhost.


Go to top




Create Certificate


Create a self-signed certificate (if you have a registered domain and already have a certificate, use your certificates)




cd ~






openssl req -x509 -newkey rsa:4096 -keyout my_matecat.key -out my_matecat.crt -days 365 -nodes (insert Country Name and Organization Name=Matecat)






mkdir /etc/apache2/ssl-cert






mv my_matecat.* /etc/apache2/ssl-cert/






mkdir /var/log/apache2/matecat




Enable virtual hosts and restart apache2




a2ensite 443-matecat.conf






a2ensite default-vhost.conf


service apache2 reload






service apache2 restart




(If you want to enable multidomain, you must create another certificate for “*.ajax.your_domain.com”)

Go to top




Configure MateCat


Go to your matecat root folder in order to install PHP composer packages:




cd /home/matecat/matecat






php -r "readfile('https://getcomposer.org/installer');" | php






php composer.phar --no-dev install






su - matecat






cd ~/matecat/inc






cp oauth_config.ini.sample oauth_config.ini






cp task_manager_config.ini.sample task_manager_config.ini






cp Error_Mail_List.ini.sample Error_Mail_List.ini






cd ~/matecat/nodejs




Paste the configuration in nodejs/config.ini file:




vi /home/matecat/matecat/nodejs/config.ini






[server]
address = 0.0.0.0
port = 7788
path = /channel/updates

[queue]
name = /topic/matecat_sse_notifications
host = localhost

port = 61613
login = admin
passcode = admin

[log]
file = log/server.log
level = debug

[cors] 
allowedOrigins[] = *




Please note that in this example there is a section dedicated to CORS, please feel free to ignore it if you don’t need it.


Go to top


MateCat Filters API


Matecat Filters is a Translated API to extract translatable text from documents in several formats as XLIFF and to generate the translated document in the same format from the translated XLIFF. The API is made available through the RapidAPI marketplace.


To make your Matecat instance use the Matecat Filters API you should sign-up on RapidAPI at https://rapidapi.com/.


The Matecat Filters API is available on RapidAPI Hub at https://rapidapi.com/translated/api/matecat-filters/ (you can also search for matecat).


Once you are registered and logged in you should obtain your RapidAPI key.


When you get your API key you can use it in the config.ini file to fill the FILTERS_RAPIDAPI_KEY value (see below).


Config File


Paste the configuration in the Matecat config.ini file:




vi /home/matecat/matecat/inc/config.ini






ENV = "production"
CHECK_FS = no
[production]
CLI_HTTP_HOST = "https://your_domain.com"
COOKIE_DOMAIN = "your_domain.com"
ENABLE_MULTI_DOMAIN_API = no
DB_SERVER = "localhost"
DB_DATABASE = "matecat"
DB_USER = "matecat"
DB_PASS = "matecat01"
REDIS_SERVERS = "tcp://localhost:6379"
QUEUE_BROKER_ADDRESS = "tcp://localhost:61613"
QUEUE_DQF_ADDRESS = "tcp://localhost:61613"
QUEUE_JMX_ADDRESS = "http://localhost:8161"
QUEUE_CREDENTIALS = "admin:admin"
SSE_BASE_URL = "your_domain.com/sse" 
STORAGE_DIR = "/home/matecat/matecat/storage" 
SMTP_HOST = 'localhost'
SMTP_PORT = '25'
SMTP_SENDER = 'no-reply@matecat.loc'
SMTP_HOSTNAME = 'localhost'
; Filters Configuration
FILTERS_ADDRESS = 'https://translated-matecat-filters-v1.p.rapidapi.com'
FILTERS_RAPIDAPI_KEY = "your RapidAPI Key"
; THIS SHOULD BE YOUR LEXIQA LICENSE, Request your license key at
; @see http://www.lexiqa.net
LXQ_LICENSE = false
LXQ_PARTNERID = false
FILE_STORAGE_METHOD = 'fs'
#TRACKING_CODES_VIEW_PATH = "internal_scripts/tracking_codes/"





Please pay particular attention to your domain URL, this MUST match your domain.


Then:




sed -i "s/\/home\/matecat\/storage/\/home\/matecat\/matecat\/storage/g" config.ini


exit




Change owner files:




chown -R matecat:matecat /home/matecat/matecat/*




Go to top




Compile JS code


Install yarn:




curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -


echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list


sudo apt-get update


sudo apt-get install yarn




Install grunt:




yarn global add grunt-cli




Install vendors and compile assets:




yarn install


grunt deploy




Open your browser and check if you see the homepage running.


Check also if there is file called secret_login.dat in /home/matecat/matecat/inc folder.


Go to top




Run Daemons


Launch analysis daemon:




cd /home/matecat/matecat/daemons






/bin/bash /home/matecat/matecat/daemons/restartAnalysis.sh






chown -R www-data /home/matecat/matecat/storage/






service apache2 restart






cd /home/matecat/matecat/nodejs/






npm install sse-channel






npm install




Run Node.js server in a screen with the following command:




screen -d -m -S 'node' node /home/matecat/matecat/nodejs/server.js




Go to top




Port


Make sure this server can be seen from the outside, in particular that ports 80 and 443 are open




HTTP TCP 80 0.0.0.0/0
HTTPS TCP 443 0.0.0.0/0





Go to top




Set Hosts


If your domain is not registered, add the ip to your hosts file




vi /etc/hosts




Add a new line with the ip and domain:




ip_server your_domain.com




Example:




127.0.0.1 localhost dev.matecat.com




Go to top