docker, creating and deploying the dba docker-image
Friday, 26 November 2021 · 7 messages · last reply 29 Nov 2021
RS
Rob Stapper26 Nov 2021 14:50 #
Hi Christian,
I’m fiddling with docker-images and containers now and I’m trying to build an deploy rhe dba-docker axample shipped with the basex-distribution.
Docker is running under windows-10. So far I’ve been able to pull, build, deploy images and containers. It all work fine but I got stuck with the dba-example.
I’ve mounted ‘c:/Program Files (x86)/basex/data’ on ‘/srv/basex/data’
But I’m confused what directories to mount on: ‘/srv/basex/repo’ and ‘/srv/basex/webapp’? I can’t find the docker-run example for this.
Can you shine some more light on the issue.
Thanx in advance,
Rob Stapper
Sent from Mail for Windows
--
This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus
CG
Christian Grün26 Nov 2021 14:57 #
Hi Rob,
I haven't worked with Docker by myself too much, but maybe other users on the list will be able to help you here.
Hartelijke groet,
Christian
Quoted text · Rob Stapper · 50 lines
Rob Stapper <r.stapper at lijbrandt.nl> schrieb am Fr., 26. Nov. 2021, 14:51: > Hi Christian, > > > > I’m fiddling with docker-images and containers now and I’m trying to build > an deploy rhe dba-docker axample shipped with the basex-distribution. > > > > Docker is running under windows-10. So far I’ve been able to pull, build, > deploy images and containers. It all work fine but I got stuck with the > dba-example. > > > > I’ve mounted ‘c:/Program Files (x86)/basex/data’ on ‘/srv/basex/data’ > > > > But I’m confused what directories to mount on: ‘/srv/basex/repo’ and > ‘/srv/basex/webapp’? > > I can’t find the docker-run example for this. > > > > Can you shine some more light on the issue. > > > > Thanx in advance, > > > > Rob Stapper > > > > Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for > Windows > > > > > <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient&utm_term=icon> Virus-free. > www.avast.com > <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient&utm_term=link> > <#m_-8195415343893423977_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> >
OS
Hi Rob,
What do you actually want to achieve here?
If you want to compile xquery files to protect your IP I have no idea how to do that. Saxon has such a feature in a limited way if you buy PE or EE if I remeber correctly. You can call Saxon using
xslt:transform.
I do a lot of docker stuff and I
* used a JVM image and mounted everything BaseX (so the zip distribution including any xquery code I added)
* tried to create a buildpack that downloads everything needed and creates a container form that:
https://github.com/simar0at/heroku-buildpack-basex
This is a https://github.com/gliderlabs/herokuish buildpack (usabel with gitlab CI/CD).
It is still rough around the edges
Maybe you want something in between?
You probably will need to mount <basexHome>/data (which also contains the logs), and <basexHome>/webapp containing any code for the (rest based?) services you want to provide.
Best regards
OmarMS
Hi Rob,
this answer contains two parts, but tldr: if you mount nothing to /srv/basex/webapp the DBA should already be present.
Now the slightly longer part:
Quoted text · Rob Stapper · 4 lines
> But I’m confused what directories to mount on: ‘/srv/basex/repo’ and ‘/srv/basex/webapp’? > I can’t find the docker-run example for this. > > Can you shine some more light on the issue.
Inside the docker image, the home-folder is /srv/basex, hence:
The /srv/basex/repo Folder contains all XQuery modules that can be imported from your query files, using the BaseX module resolution.
From the documentation:
Quoted text · Rob Stapper · 1 line
> Points to the Repository, in which all XQuery modules are located.
https://docs.basex.org/wiki/Repository <https://docs.basex.org/wiki/Repository>
The webapp folder on the other hand contains your RestXQ-annotated-XQuery files, and is configured via the
WEBPATH property:Quoted text · Rob Stapper · 4 lines
> WEBPATH > Signature WEBPATH [path] > Default {home}/webapp > Summary Points to the directory in which all the Web Application contents are stored, including XQuery, Script, RESTXQ and configuration files
As the docker container is (more or less) the ZIP-File extracted to a specific location, it behaves very much like starting basexhttp from the commandline.
When you download the BaseX Zipfile, you run:
Quoted text · Rob Stapper · 1 line
> bin/basexhttp
Which in turn runs: java -cp ${all-basex-libraries-and-jarfiles} org.basex.BaseXHTTP
The BaseXHTTP-process now looks for the given folders in the current home directory:
-
data => for databases
- repo => for XQuery modules
- webapp => for web applications
When it comes to docker, the home directory is /srv/basex, with the following layout:
/srv/basex/data => for databases
/srv/basex/repo => for XQuery modules
/srv/basex/webapp => for RestXQ-annotated stuff such as the dba
As the ZIP-File contains the DBA, all needed files should be already present, if you did not mount something else into /srv/basex/webapp.
If you want to add your own RestXQ endpoints for example, the safe way is to mount them into a subfolder of /srv/basex/webapp, for example:Quoted text · Rob Stapper · 7 lines
> docker run -d \ > --name basexhttp \ > --publish 1984:1984 \ > --publish 8984:8984 \ > --volume "$HOME/basex/data":/srv/basex/data \ > --volume "$(pwd)/webapp/myapp":/srv/basex/webapp/myapp \ > basex/basexhttp:latest
That way the webapp folder in the docker container keeps the DBA and also has your files present.
If you plan to distribute your image I’d suggest building your own image with a Dockerfile. I created a minimal example here: https://git.basex.io/basex-org/basex-app-with-docker <https://git.basex.io/basex-org/basex-app-with-docker> All I’ve done is: download BaseX.zip, extract BaseX.zip, add a Dockerfile to build the image.
Hope this helps :-)
Michael
Quoted text · Rob Stapper · 5 lines
> Am 26.11.2021 um 14:50 schrieb Rob Stapper <r.stapper at lijbrandt.nl>: > > I’ve mounted ‘c:/Program Files (x86)/basex/data’ on ‘/srv/basex/data’ > >
RS
Rob Stapper27 Nov 2021 16:39 #
Hi Michael,
Thanx for the extended answer. Eventually I got it working with my web application and xquey-modules as part of the image. I had a little trouble with the ownerships of the mounted datafiles but ‘chown’ under the ‘root’ solved that one. I find the image size a pretty big: 400Mb against the 227 Mb of the regular ‘basex/basexhttp’-image. On the other hand the performance is much better then a basexhttp-container with my code mounted on container-creation. I’m content for now, I’ll grab a beer.
Cheers,
Rob
Sent from Mail for Windows
Quoted text · Michael Seiferle · 53 lines
From: Michael Seiferle Sent: Friday, November 26, 2021 6:04 PM To: Rob Stapper Cc: basex-talk at mailman.uni-konstanz.de Subject: Re: [basex-talk] docker, creating and deploying the dba docker-image Hi Rob, this answer contains two parts, but tldr: if you mount nothing to /srv/basex/webapp the DBA should already be present. Now the slightly longer part: But I’m confused what directories to mount on: ‘/srv/basex/repo’ and ‘/srv/basex/webapp’? I can’t find the docker-run example for this. Can you shine some more light on the issue. Inside the docker image, the home-folder is /srv/basex, hence: The /srv/basex/repo Folder contains all XQuery modules that can be imported from your query files, using the BaseX module resolution. From the documentation: Points to the Repository, in which all XQuery modules are located. https://docs.basex.org/wiki/Repository The webapp folder on the other hand contains your RestXQ-annotated-XQuery files, and is configured via the WEBPATH property: WEBPATH Signature WEBPATH [path] Default {home}/webapp Summary Points to the directory in which all the Web Application contents are stored, including XQuery, Script, RESTXQ and configuration files As the docker container is (more or less) the ZIP-File extracted to a specific location, it behaves very much like starting basexhttp from the commandline. When you download the BaseX Zipfile, you run: bin/basexhttp Which in turn runs: java -cp ${all-basex-libraries-and-jarfiles} org.basex.BaseXHTTP The BaseXHTTP-process now looks for the given folders in the current home directory: - `data` => for databases - `repo` => for XQuery modules - `webapp` => for web applications When it comes to docker, the home directory is /srv/basex, with the following layout: /srv/basex/data => for databases /srv/basex/repo => for XQuery modules /srv/basex/webapp => for RestXQ-annotated stuff such as the dba As the ZIP-File contains the DBA, all needed files should be already present, if you did not mount something else into /srv/basex/webapp. If you want to add your own RestXQ endpoints for example, the safe way is to mount them into a subfolder of /srv/basex/webapp, for example: docker run -d \ --name basexhttp \ --publish 1984:1984 \ --publish 8984:8984 \ --volume "$HOME/basex/data":/srv/basex/data \ --volume "$(pwd)/webapp/myapp":/srv/basex/webapp/myapp \ basex/basexhttp:latest That way the webapp folder in the docker container keeps the DBA and also has your files present. If you plan to distribute your image I’d suggest building your own image with a Dockerfile. I created a minimal example here: https://git.basex.io/basex-org/basex-app-with-docker All I’ve done is: download BaseX.zip, extract BaseX.zip, add a Dockerfile to build the image. Hope this helps :-) Michael Am 26.11.2021 um 14:50 schrieb Rob Stapper <r.stapper at lijbrandt.nl>: I’ve mounted ‘c:/Program Files (x86)/basex/data’ on ‘/srv/basex/data’ -- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus
RS
Yes, and my angular-frontend connects nicely, though a little slow, with the container.
Cheers,
Rob
Sent from Mail for Windows
Quoted text · Michael Seiferle · 53 lines
From: Michael Seiferle Sent: Friday, November 26, 2021 6:04 PM To: Rob Stapper Cc: basex-talk at mailman.uni-konstanz.de Subject: Re: [basex-talk] docker, creating and deploying the dba docker-image Hi Rob, this answer contains two parts, but tldr: if you mount nothing to /srv/basex/webapp the DBA should already be present. Now the slightly longer part: But I’m confused what directories to mount on: ‘/srv/basex/repo’ and ‘/srv/basex/webapp’? I can’t find the docker-run example for this. Can you shine some more light on the issue. Inside the docker image, the home-folder is /srv/basex, hence: The /srv/basex/repo Folder contains all XQuery modules that can be imported from your query files, using the BaseX module resolution. From the documentation: Points to the Repository, in which all XQuery modules are located. https://docs.basex.org/wiki/Repository The webapp folder on the other hand contains your RestXQ-annotated-XQuery files, and is configured via the WEBPATH property: WEBPATH Signature WEBPATH [path] Default {home}/webapp Summary Points to the directory in which all the Web Application contents are stored, including XQuery, Script, RESTXQ and configuration files As the docker container is (more or less) the ZIP-File extracted to a specific location, it behaves very much like starting basexhttp from the commandline. When you download the BaseX Zipfile, you run: bin/basexhttp Which in turn runs: java -cp ${all-basex-libraries-and-jarfiles} org.basex.BaseXHTTP The BaseXHTTP-process now looks for the given folders in the current home directory: - `data` => for databases - `repo` => for XQuery modules - `webapp` => for web applications When it comes to docker, the home directory is /srv/basex, with the following layout: /srv/basex/data => for databases /srv/basex/repo => for XQuery modules /srv/basex/webapp => for RestXQ-annotated stuff such as the dba As the ZIP-File contains the DBA, all needed files should be already present, if you did not mount something else into /srv/basex/webapp. If you want to add your own RestXQ endpoints for example, the safe way is to mount them into a subfolder of /srv/basex/webapp, for example: docker run -d \ --name basexhttp \ --publish 1984:1984 \ --publish 8984:8984 \ --volume "$HOME/basex/data":/srv/basex/data \ --volume "$(pwd)/webapp/myapp":/srv/basex/webapp/myapp \ basex/basexhttp:latest That way the webapp folder in the docker container keeps the DBA and also has your files present. If you plan to distribute your image I’d suggest building your own image with a Dockerfile. I created a minimal example here: https://git.basex.io/basex-org/basex-app-with-docker All I’ve done is: download BaseX.zip, extract BaseX.zip, add a Dockerfile to build the image. Hope this helps :-) Michael Am 26.11.2021 um 14:50 schrieb Rob Stapper <r.stapper at lijbrandt.nl>: I’ve mounted ‘c:/Program Files (x86)/basex/data’ on ‘/srv/basex/data’ -- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus
MS
Hi Rob,
Glad you got it up and running :)
Quoted text · Rob Stapper · 5 lines
> Am 27.11.2021 um 16:39 schrieb Rob Stapper <r.stapper at lijbrandt.nl>: > > Hi Michael, > > Thanx for the extended answer. Eventually I got it working with my web application and xquey-modules as part of the image. I had a little trouble with the ownerships of the mounted datafiles but ‘chown’ under the ‘root’ solved that one.
Yeah, I think the obvious solution to this would be running the entire image as root — which may or may not impose a security issue in your architecture.
Quoted text · Rob Stapper · 1 line
> I find the image size a pretty big: 400Mb against the 227 Mb of the regular ‘basex/basexhttp’-image.
Yeah I think you could boil this down to a few megabytes (using the alpine variant) — but I moved to arm (thanks Apple M1) recently there is no OpenJDK / Alpine Image yet, hence the size.
Quoted text · Rob Stapper · 1 line
> On the other hand the performance is much better then a basexhttp-container with my code mounted on container-creation. I’m content for now, I’ll grab a beer.
Just a guess, but maybe that performance plus stems from the use of different JDKs? My example contains JDK 11 — the official BaseX Docker is running JDK 8. I think we might consider updating our official image.
Best
Michael
Quoted text · Rob Stapper · 186 lines
> > Cheers, > Rob > > Sent from Mail for Windows > > From: Michael Seiferle > Sent: Friday, November 26, 2021 6:04 PM > To: Rob Stapper > Cc: basex-talk at mailman.uni-konstanz.de > Subject: Re: [basex-talk] docker, creating and deploying the dba docker-image > > > Hi Rob, > > > this answer contains two parts, but tldr: if you mount nothing to /srv/basex/webapp the DBA should already be present. > > > > Now the slightly longer part: > > > > > > But I’m confused what directories to mount on: ‘/srv/basex/repo’ and ‘/srv/basex/webapp’? > > > > I can’t find the docker-run example for this. > > > > > > > > Can you shine some more light on the issue. > > > > > > > Inside the docker image, the home-folder is /srv/basex, hence: > > > > The /srv/basex/repo Folder contains all XQuery modules that can be imported from your query files, using the BaseX module resolution. > > > > From the documentation: > > > > Points to the Repository, in which all XQuery modules are located. > https://docs.basex.org/wiki/Repository > > > > The webapp folder on the other hand contains your RestXQ-annotated-XQuery files, and is configured via the WEBPATH property: > > > > > > WEBPATH > > Signature WEBPATH [path] > > Default {home}/webapp > > Summary Points to the directory in which all the Web Application contents are stored, including XQuery, Script, RESTXQ and configuration files > > > > > As the docker container is (more or less) the ZIP-File extracted to a specific location, it behaves very much like starting basexhttp from the commandline. > > > > When you download the BaseX Zipfile, you run: > > > > bin/basexhttp > Which in turn runs: java -cp ${all-basex-libraries-and-jarfiles} org.basex.BaseXHTTP > > > > The BaseXHTTP-process now looks for the given folders in the current home directory: > > > > - `data` => for databases > > - `repo` => for XQuery modules > > - `webapp` => for web applications > > > > > When it comes to docker, the home directory is /srv/basex, with the following layout: > > > > /srv/basex/data => for databases > > /srv/basex/repo => for XQuery modules > > /srv/basex/webapp => for RestXQ-annotated stuff such as the dba > > > > > > As the ZIP-File contains the DBA, all needed files should be already present, if you did not mount something else into /srv/basex/webapp. > > > > If you want to add your own RestXQ endpoints for example, the safe way is to mount them into a subfolder of /srv/basex/webapp, for example: > > > > > > docker run -d \ > > > > --name basexhttp \ > > > > --publish 1984:1984 \ > > > > --publish 8984:8984 \ > > > > --volume "$HOME/basex/data":/srv/basex/data \ > > > > --volume "$(pwd)/webapp/myapp":/srv/basex/webapp/myapp \ > > > > basex/basexhttp:latest > > > > > That way the webapp folder in the docker container keeps the DBA and also has your files present. > > > > > > If you plan to distribute your image I’d suggest building your own image with a Dockerfile. > > I created a minimal example here: https://git.basex.io/basex-org/basex-app-with-docker > > All I’ve done is: download BaseX.zip, extract BaseX.zip, add a Dockerfile to build the image. > > > > > > > > > > Hope this helps :-) > > > > > > > > Michael > > > > > > > > > > > > Am 26.11.2021 um 14:50 schrieb Rob Stapper <r.stapper at lijbrandt.nl>: > > > > > > I’ve mounted ‘c:/Program Files (x86)/basex/data’ on ‘/srv/basex/data’ > > > > > > > > > > > > > > > Virus-free. www.avast.com
Written for BaseX 9.6. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery