Official binary distributions of the Martian toolchain are available for Linux. We hope to have MacOS and Windows support soon.
- Currently, running pipelines requires a Linux system with kernel version 2.6.23 or later.
- Martian 2.x has been confirmed working on Linux versions as far back as RHEL/CentOS 5.5 or Ubuntu 10.
- Martian 3.0 and higher are supported on RHEL/CentOS 6 or Ubuntu 12 or higher.
- For the time being,
mrponly functions on Linux.
- Ancillary tools such as
mrstatare expected to work on MacOS.
- Martian was designed for large bioinformatics pipelines. One may run into issues on systems with less than 6GB of available memory. Editing
jobmanagers/config.jsonto reduce the default memory request for jobs may alleviate some of those issues.
Building from Source
- Go 1.18 or higher is required to build Martian.
- The Python adapter for wrapping stage code requires Python 2.7 (until version 4.0.9) or 3.6 or higher.
- To build the user interface, Node 14 or higher is required, along with
Building the source
To build the Martian toolchain from source, clone the Martian GitHub repository, run
make all, and the compiled binaries will be generated in
$ git clone --recursive https://github.com/martian-lang/martian.git $ cd martian $ make all $ ls bin mrjob mro mrp mrstat
To test that everything is working,
make longtests runs a few simple test pipelines
and verifies that their output is correct, including tests that pipeline failures are
Clone the repository and run
bazel test //...
Build with go
You can use
$ go get golang.org/x/tools/cmd/goyacc $ go install golang.org/x/tools/cmd/goyacc $ go get github.com/martian-lang/martian/cmd/... $ go generate github.com/martian-lang/martian/... $ go install github.com/martian-lang/martian/cmd/...
However that will not you’ll still need to run
make web in the repository
directory in order to build the web UI. Furthermore, the martian binaries
expect various data files to be located in specific relative locations.
make is therefore recommended.
The Martian toolchain comprises five core executables:
|Compiler/Checker||Parses and validates Martian code|
|Formatter||Canonicalizes Martian code formatting and whitespace|
|Analysis||Supports queries, e.g. tracing inputs/outputs, as well as formatting the call graph.|
|Refactoring||Rename or remove stages/pipelines/inputs/outputs.|
|Pipeline Runtime||Executes a Martian pipeline or stage.|
|Stage wrapper||Wraps user stage code, ensuring it obeys the contracts |
|Progress query||CLI tool for querying |
Make these executables available on your
PATH and then confirm that you can run them. If you unpacked or cloned Martian into
/home/user/git/martian, for example, then:
$ export PATH=$PATH:/home/user/git/martian/bin $ mro --version v4.0.2
Martian Project Path – MROPATH
The code for a Martian pipeline project typically lives under a single directory. Set
MROPATH to this directory to allow the Martian executables to find the project code without the need for absolute paths.
When developing multiple projects, switch between them by changing
To give you an idea of how a Martian project looks in practice, here’s an example:
martian_project/ bin/ hello_go hello_rust lib/ go/ vendor/github.com/martian-lang/martian src/martian/adapter/adapter.go src/ hello_go.go rust/ Cargo.lock Cargo.toml src/ hello_rust.rs mro/ _hello_stages.mro hello.mro stages/ hello/ hello_py/ __init__.py
Stage code (or binaries) are searched for relative to
* the mro file defining the stage