The recommended way to develop stanc3 is using opam and dune on a Unix-like system (MacOS or Linux). See Windows development for tips on developing on Windows.
This requires you have make, git, curl, and m4 installed via apt, homebrew, or your system's package manager.
Install the necessary dependencies by running
cd scripts; bash -x ./setup_dev_env.sh.
If you already have opam installed, you can instead run
opam update; opam install --deps-only --with-test .
If you wish to install stanc on your system, you can run the above opam command without --deps-only.
make. If this succeeds, you have the required software.We recommend using Windows Subsystem for Linux (WSL) for development on Windows. While this will not produce native-Windows binaries by default (though they can be cross-compiled), it is far easier to use and set up than the other options for Windows development.
Once you have installed and configured WSL, you can proceed through the steps above through the WSL shell.
Nix is a declarative package manager with a focus on reproducible builds. We provide the ability to use Nix to build, test and run Stanc3. We recommend trying the opam instructions first if you are not an existing Nix user, with these as a backup.
If you have nix installed, you can build Stanc3 by running the following command in the stanc3 directory:
nix-build
The binary will be in result/bin/stanc. It may take a minute the first time you run it. Alternatively, the following is sometimes a faster way to build:
nix-shell --command "dune build"
To run the test suite, run:
nix-shell --command "dune build --profile release @runtest"
To install Stanc3 to your system, run:
nix-env -i -f default.nix
To drop into a sandboxed development shell with all of the build dependencies of Stanc3 plus packages for an OCaml development environment (dune, ocp-indent, ocamlformat, merlin and utop), run:
nix-shell
Once you have the necessary software, you can make your edits to the source files in src/ and tests in test/. Running the command dune runtest (or make test) will show any test output which is different from the expected output. In our continuous integration, this results in test failures. If you are knowingly changing the output, and the differences highlighted by dune runtest look correct, you can run dune promote to change the expected output to match your current tests.
We have support for computing test coverage using the bisect_ppx library. To generate a coverage report, run make testcoverage. This will print a summary of the coverage to the terminal, and generate a detailed HTML report in the _coverage directory. This report can be opened in a web browser.
Code coverage of the master branch is available on codecov.io.
We recommend setting up your editor to use OCamlformat automatically. This will differ by editor, but is possible in most popular editors such as Emacs, VSCode, etc. See Editor advice for more specific tips.
If you would like to manually run the auto-formatter on all OCaml code in the repository, run make format. Similar to the above, you will need to run dune promote to accept these changes into the code.
If you would like to run the formatter before each commit (recommended), run
scripts/hooks/install_hooks.sh
Note: ocamlformat does not support .mly or .mll files used in the frontend.
If the parser has been edited, the command dune build @update_messages will add any new error states to the parser.messages file. These will say
TODO: PARSER MESSAGE NEEDED HERE.
See adding or changing the syntax error messages for more information.
Note that python3 is required to run this command.
Docs can be built with make doc. This will generate a _build/default/_doc/_html/index.html file which can be opened in a web browser.
In order to use the locally built development version of stanc3 with CmdStan, you need to set the STANC3 makefile variable in the make/local file in CmdStan. Example:
STANC3=relative/or/absolute/path/to/the/cloned/stanc3/folder
This will automatically rebuild the stanc3 binary used in CmdStan if a change was made to the stanc3 source files.
For working on this project, we recommend using either VSCode or Emacs as an editor, due to their good OCaml support through Merlin: syntax highlighting, auto-completion, type inference, automatic case splitting, and more. For people who prefer a GUI and have not memorized all Emacs or Vim keystrokes, VSCode might have the less steep learning curve. Anything with Merlin support is fine.
Install instructions for VSCode can be found here.
For Windows users: We recommend using Remote - WSL to program within WSL from a native Windows installation of VSCode. This extension allows you to connect to the WSL instance as if it is a remote machine, removing the need for an X11 server or other method of running a Linux GUI app directly.
Once in VSCode (on any platform), simply install the OCaml extension and you should be ready to go.
The best way to edit OCaml in Emacs is through Tuareg mode.
To get full Merlin support, you will need to run opam user-setup install after installing Merlin in our dev dependencies.
If you use use-package.el, this snippet should set up Tuareg+Merlin+OCamlformat
(use-package tuareg
:config
(electric-indent-mode 0)
(face-spec-set
'tuareg-font-lock-constructor-face
'((((class color) (background light)) (:foreground "SaddleBrown"))
(((class color) (background dark)) (:foreground "burlywood1"))))
(add-hook 'tuareg-mode-hook
(lambda ()
(prettify-symbols-mode 0))))
(use-package merlin
:config
(add-hook 'tuareg-mode-hook 'merlin-mode t)
(with-eval-after-load 'company
(add-to-list 'company-backends 'merlin-company-backend)))
(require 'ocamlformat)
(add-hook 'tuareg-mode-hook
(lambda ()
(define-key tuareg-mode-map (kbd "C-M-<tab>") #'ocamlformat)
(add-hook 'before-save-hook #'ocamlformat-before-save)))Even to an experienced OCaml developer, there are certain practices in the stanc3 codebase which might be unfamiliar. We have tried to document those on our Core ideas page.
OCaml has support for building binaries instrumented for use with theAFL++ fuzzing tool. This requires a new opam switch which includes the ocaml-option-afl base package.
opam switch create "stanc-fuzz" --packages "ocaml-option-afl,ocaml.4.14.1"
eval $(opam env)
./scripts/install_build_deps.shAfter this, the normal build process will produce a binary ready for fuzzing.
Profiling support is not directly provided in the stanc3 repository, but you can use the Landmarks library to profile the code.
This requires editing the dune files in the src/ directory to include the landmarks library, similar to how the bisect_ppx library is instrumented by default.
(instrumentation
(backend landmarks --auto))You can then build a profiled version of the code by running dune build src/stanc/stanc.exe --instrument-with landmarks --force.
To configure the profiler at runtime, consult the Landmarks documentation. A good starting command is
OCAML_LANDMARKS="on,output=prof.txt" stanc.exe mymodel.stan