Build the package
Build the package with Nimble (included with Nim). The tool takes care of dependencies:
$ nimble build covidphy
$ nimble jvs
Add a .env file
The covidphy
binary looks for a file .env
in the same dir. Create this file
readable only by owner to configure your http server:
$ touch .env
$ chmod 600 .env
Edit the file to include the following adjustable settings:
debug=false
bindAddr=0.0.0.0
port=8080
appName=covidphy.eu
staticDir=/static
alignDir=/aligns
logLevel=lvlDebug
mailFrom="[email protected]"
mailTo="[email protected]"
mail=YourMailPassword
secretKey=A_Long_Random_Chain_Of_Chars
db=db.sqlite3
entrezKey=The_Entrez_Key
Setup the DB
The DB should be created empty by running Jester the first time. But tables are not (by now). Build the tables with the following script:
$ nim r src/migrate.nim all
Load the needed fixtures from src/fixtures/name.sql
:
$ cat src/fixtures/name.sql | sqlite3 db.sqlite3 # One fixture
$ for F in src/fixtures/*sql; do cat $F | sqlite3 db.sqlite3; done #* All at once
The variant frequency data can be loaded with a script:
$ nim r src/helpers/updateVariantsServer src/fixtures/variant.sql
The web includes a page to search time evolution of groups of variants. This requires a huge DB that's out of the repo, that is configured at your .env
file with:
haplotypeDb="src/resources/haplotype.sqlite3"
As with the db
env variable, the path is relative to the binary covidphy
compiled in the first step.
The schema of the database is a single virtual table defined as CREATE VIRTUAL TABLE "haplotype" USING fts5(id UNINDEXED, mutations, sampled UNINDEXED, country UNINDEXED)
, and a view for counting defined as CREATE VIEW day_totals AS SELECT sampled, count(*) AS totals FROM haplotype GROUP BY sampled
. The data comes as:
id | mutations | date_sampled | country |
---|---|---|---|
1 | 2019-12-30 | China | |
2 | C27493T C28253T | 2020-01-01 | USA |
3 | G20670A G20679A | 2019-12-30 | United Kingdom |
4 | A3778G A8388G T8987A | 2019-12-24 | Spain |
We cannot distribute this table because of the Gisaid restrictions: try to build one yourself, ask us for help if needed.
Make a log dir
A log dir is not strictly necessary, but if you deploy the server you might want to collect the logs together. Fossil is instructed to ignore the path "log/*.log".
Running the server
The server is ready to run. Launch it with:
$ ./covidphy
It won't say anything (maybe Serving at 0.0.0.0:8080
). You can test it worked
by curling localhost at the port configured in the .env file above. Open
another terminal and type:
$ curl 127.0.0.1:8080
<!DOCTYPE html>
...
</html>
Configure nginx to listen
It's a bad practice to expose servers directly to the internet. Use a proxy like Nginx to transfer the requests to the application. This goes well beyond the aim of this document, but it can be resumed as:
O1: oval "User" fit; B1: box at 2cm right of O1 "Nginx" fit; B2: box at 2cm right of B1 "CovidPhy" fit; O2: oval at 2cm right of B2 "Dynamic" fit; O3: oval with .c at 1cm south of O2.s "Static" fit; arrow <-> from O1.e to B1.w; arrow <-> from B1.e to B2.w; arrow <-> from B2.e to O2.w; arrow <-> from B1.s down 1cm then to O3.w;
Create a
server
directive that points/
to aproxy_pass
:server { location / { proxy_pass http://covidphy/; } }
Read some docs to set the parameters proxy_set_header
, proxy_redirect
and
others needed.
Create an
upstream
pointing to the server compiled above. Depending on .env:upstream covidphy { server 127.0.0.1:8080; }
Fix the
static
dir to be server directly by Nginx instead of the server:location /static/ { alias /path/of/your/static/; }
This is not complicated, but it's also not for beginners. Look for help. It can be done in minutes if you know what your are doing, but takes days if you don't.
Some other gotchas to nginx configuration:
Run nginx as proper user.
Ensure
/var/lib/nginx
permissions are OK:# chown -R userabove:nginx /var/lib/nginx
Configure supervisor to do its thing
Note: this can be done also with systemd.
Use supervisor to launch the command. The config .ini file include at least the following entries: directory, command, user and stdout_logfile.
[program:covidphy]
directory = /abs/path/to/covidphy.eu/
command = /abs/path/to/covidphy.eu/covidphy ; Command to start app
user = genvip ; User to run as
stdout_logfile = /abs/path/to/covidphy.eu/log/supervisor.log ; Where to write log messages
redirect_stderr = true ; Save stderr in the same log
environment=LANG=en_US.UTF-8,LC_ALL=en_US.UTF-8 ; Set UTF-8 as default encoding
Then reload the daemon:
# supervisorctl reread
covidphy: available
# supervisorctl avail
covidphy avail auto 999:999
# supervisorctl update
Manteinance
From the deployment dir:
$ fossil up
$ nim c -d:danger -o:covidphy src/covidphy
$ nimble jvs
### If needed ###
$ nim r src/migrate XXXX
$ cat src/fixtures/name.sql |sqlite3 db.sqlite3
### The variant frequency data can only be loaded with a script:
$ nim r src/helpers/updateVariantServer.nim src/fixtures/variant.sql
### Restart ###
$ sudo supervisorctl restart covidphy