Module `Topkg.Pkg`

Package description.

See the basics.

Installation description

The installation description generates an opam install file which is simply a description of file moves (in the mv sense) from the build or source directory to standard install directories. Describing these moves in a given build configuration effectively determines what needs to built by the package build command.

Note. Always use "/" as a directory seperator for paths, even on Windows.

type install: The type for representing a set of install moves.

val nothing : install: nothing is an empty set of install moves.

val flatten : install list -> install: flatten installs is the union of all the install moves in installs.

type field = ?⁠force:bool -> ?⁠built:bool -> ?⁠cond:bool -> ?⁠exts:Exts.t -> ?⁠dst:fpath -> fpath -> install

The type for an install field, a function that describe file moves to a particular installation directory. In the simplest form a call field src simply moves the file src at the root of the install directory of the field.

In general field ~force ~built ~cond ~exts ~dst src generates install move as follows:

dst is the path were the source is written to. Expressed relative to the install directory of the field. Defaults to Fpath.basename src, i.e. at the root of the install directory. If dst is a directory path, the destination is (dst ^ Fpath.basename src).
If exts is present and non empty, generates the list of paths List.map (fun e -> src ^ e) and a move for each of these. For example field ~exts:Exts.api "src/m" would generate a move for the files "src/m.mli", "src/m.cmi", "src/m.cmti", "src/m.cmx".
If cond is false (defaults to true) no file move is generated. This provides a convenient way to conditionalize installation based on the build configuration for example:
```
let jsoo = Conf.value jsoo in
Pkg.mllib ~cond:jsoo "src/mylib_jsoo.mllib"
```
If built is true (default), src is expressed relative to the build directory of the distribution and the path src is be given to build system invocation for construction. If false, src is relative to the root of the distribution and is excluded from the build system invocation; this can be used for installing files that don't need to be built.
If force is true (defaults to false) it disables the automatic src filtering performed by the library. When false, the library automatically disable certain build artefact depending on OCaml's configuration, one such example is filtering native code build artefact if the OCaml install doesn't support native code compilation

type exec_field = ?⁠auto:bool -> field

The type for fields that install executable files. This is like field except for the additional auto parameter:

If auto is true (default) then field src dst automatically adds the ".native" suffix to src if Conf.OCaml.native is true and the ".byte" suffix otherwise. Besides it automatically adds Exts.exe to dst, which handles things correctly on the various Windows ports.
If auto is false you have full control according to field.

val bin : exec_field: bin is a field that installs to a common bin/ directory.

val doc : field: doc is a field that installs to a package specific doc/ directory

val etc : field: etc is a field that installs to a package specific etc/ directory.

val lib : field: lib is a field that installs to a package specific lib/ directory.

val lib_root : field: lib_root is a field that install to a common lib/ directory.

val libexec : exec_field: libexec is a field that installs to a package specific lib/ directory but with the executable bit set.

val libexec_root : exec_field: libexec_root is a field that install to a common lib/ directory but with the executable bit set.

val man : field: man is a field that installs to a common man/ directory. See the opam manual for details.

val misc : field: misc is a field that installs to an arbitrary absolute path, the user is prompted for authorization, see the opam manual for details.

val sbin : exec_field: sbin is a field that installs to a common sbin/ directory.

val share : field: share is a field that installs to a package specific share/ directory.

val share_root : field: share_root is a field that installs to a common share/ directory.

val stublibs : field: stublibs is a field that install to a common lib/stublibs/ directory. Used for OCaml C stub directory.

val toplevel : field: toplevel is a field that installs to a common lib/toplevel/ directory.

Test executable description

val test : ?⁠run:bool -> ?⁠dir:fpath -> ?⁠args:Cmd.t -> exec_field: test is a special executable field: it doesn't install the described executable (as such the dst argument is ignored at the moment). If run is true (default) executes the test with args when ocaml pkg/pkg.ml test is run; this will notably run to test the distribution tarball. If run is false the test needs to be invoked explicitely. dir specifies the current working directory for the test, expressed relative to the root directory of the package (defaults to .).

OCamlbuild higher-level installs

The following functions are OCamlbuild specific higher level installs that generate moves by reading OCamlbuild specification files. They also automatically handle the Conf.debugger_support configuration key by building and installing the build artefacts needed by debuggers whenever its value is true.

val mllib : ?⁠field:field -> ?⁠cond:bool -> ?⁠cma:bool -> ?⁠cmxa:bool -> ?⁠cmxs:bool -> ?⁠api:string list -> ?⁠dst_dir:fpath -> fpath -> install

mllib ~field ~cond ~cma ~cmxa ~cmxs ~api ~dst_dir mllib installs an OCaml library described by the OCamlbuild .mllib file mllib with:

field is the field where it gets installed (defaults to lib).
If cond is false (defaults to true), no move is generated.
cma, cmxa, cmxs determine if corresponding archives are built and installed, they all default to true.
api is the list of modules that defines the public interface of the library, if None all the modules mentioned in mllib are part of the public interface.
dst_dir is the destination directory of the library in the field. If unspecified this is the root of the field's directory.

val clib : ?⁠dllfield:field -> ?⁠libfield:field -> ?⁠cond:bool -> ?⁠lib_dst_dir:fpath -> fpath -> install

clib clib installs C stubs described by the OCamlbuild .clib file clib with:

dllfield is the field where the C DLL archive gets installed, (defaults to stublibs)
libfield is the field where the C static archive gets installed (defaults to lib)
If cond is false (defaults to true), no move is generated.
lib_dst_dir is the destination directory of the library in the libfield field. If unspecified this is the root of the field's directory. This does not affect the dllfield, DLLs are always installed at the root directory of the dllfield.

Build description

type build: The type for package build description.

val build : ?⁠prepare_on_pin:bool -> ?⁠dir:fpath -> ?⁠pre:(Conf.t -> unit result) -> ?⁠cmd:(Conf.t -> Conf.os -> fpath list -> unit result) -> ?⁠post:(Conf.t -> unit result) -> ?⁠clean:(Conf.os -> build_dir:fpath -> unit result) -> unit -> build

build ~prepare_on_pin ~dir ~cmd ~pre ~post describes the package build procedure.

prepare_on_pin if true (default) distribution preparation is performed if a `Pin build context is detected. This means that the checkout is watermarked and the massage hook is invoked, see step 2. of distribution creation.
dir is the directory where build artefacts are generated, (defaults to "_build"). Note that his value can be overriden from the command line.
pre is a hook that is invoked with the build context, after distribution preparation if applicable, but before the build command. It can be used to adapt the build setup according to the build configuration. Default is a nop.
cmd invokes the build system to build the files determined by install moves. It is given the build configuration, an OS specification, the list of files to build relative to the build directory, and build the given files in the build directory. The default is:
```
fun c os files -> OS.Cmd.run @@ Cmd.(Pkg.build_cmd c os %% of_list files)
```
post is a hook that is invoked with the build context after the build command returned sucessfully. Default is a nop.
clean is invoked to clean a build. It is given an OS specification and a build directory to clean. The default is:
```
let clean os ~build_dir = OS.Cmd.run @@ Pkg.clean_cmd os ~build_dir
```

Warning. If you are invoking tools in your hooks consider using Tool lookup to look them up it helps for cross-compilation.

OCamlbuild support

If you are using ocamlbuild, the following functions help to customize the build system invocation according to the configuration.

val build_cmd : Conf.t -> Conf.os -> Cmd.t

build_cmd c os is the default build command to which files to build are given. Its value is defined by:

fun c os ->
  let ocamlbuild = Conf.tool "ocamlbuild" os in
  let build_dir = Conf.build_dir c in
  let toolchain =
    match Topkg_conf.toolchain c with
    | Some toolchain -> Topkg_cmd.(v "-toolchain" % toolchain)
    | _ -> Topkg_cmd.empty
  in
  let debug = Cmd.(on (Conf.debug c) (v "-tag" % "debug")) in
  let profile = Cmd.(on (Conf.profile c) (v "-tag" % "profile")) in
  Cmd.(ocamlbuild % "-use-ocamlfind" %% toolchain % "-classic-display" %%
                    debug %% profile % "-build-dir" % build_dir)

val clean_cmd : Conf.os -> build_dir:fpath -> Cmd.t

clean_cmd os ~build_dir is the default clean command. Its value is defined by:

fun os ~build_dir ->
  let ocamlbuild = Conf.tool "ocamlbuild" os in
  Cmd.(ocamlbuild % "-use-ocamlfind" % "-classic-display" %
                    "-build-dir" % build_dir % "-clean")

val ocb_tag : Conf.t -> 'a Conf.key -> string -> Cmd.t

ocb_tag c key name is a command fragment adding the ocamlbuild parameterized tag name with key's value to the default set of tags. The key value is converted to a string using the printer of the key value converter.

For example with a key k : bool Conf.key whose value is true, ocb_tag c k "foo" adds the tag foo(true) to the default tags. A sample build command for Build description using this key would be:

let cmd c os files =
  OS.Cmd.run Cmd.(build_cmd c os %% ocb_tag c k "foo" %% of_list files)

val ocb_bool_tag : Conf.t -> bool Conf.key -> string -> Cmd.t: ocb_bool_tag c key name is a command fragment adding the ocamlbuild tag name to the default set of tags iff key's value is true.

val ocb_bool_tags : Conf.t -> (bool Conf.key * string) list -> Cmd.t: ocb_bool_tags c assoc is the concatenation of ocb_bool_tag for all pairs in assoc.

Distribution description

The type for watermarks. A watermark identifier, e.g. "ID" and its definition:

`String s, s is the definition.
`Name, is the name of package.
`Version, is the version of the distribution.
`Version_num, is the version of the distribution with a potential leading 'v' or 'V' dropped.
`Vcs `Commit_id, is the commit identifier (hash) of the distribution. May be post-fixed by "dirty" in dev package (`Pin) builds.
`Opam (file, field, sep), is the values of the field field concatenated with separator sep of the opam file file, expressed relative to the distribution root directory, if file is None this is the package's default opam file, see describe. Not all fields are supported see the value of Topkg_care.Opam.File.field_names. Warning. In dev package (`Pin) builds, `Opam watermarks are only substituted if the package topkg-care is installed.

When a file is watermarked with an identifier "ID", any occurence of the sequence %%ID%% in its content is substituted by its definition.

type distrib: The type for describing distribution creation.

val distrib : ?⁠watermarks:watermark list -> ?⁠files_to_watermark:(unit -> fpath list result) -> ?⁠massage:(unit -> unit result) -> ?⁠exclude_paths:(unit -> fpath list result) -> ?⁠uri:string -> unit -> distrib

distrib ~watermarks ~files_to_watermark ~massage ~exclude_paths ~uri () influences the distribution creation process performed by the topkg tool. See the full details about distribution creation.

In the following the distribution build directory is a private clone of the package's source repository's HEAD when topkg distrib is invoked.

watermarks defines the source watermarks for the distribution, defaults to watermarks.
files_to_watermark is invoked in the distribution build directory to determine the files to watermark, defaults to files_to_watermark.
massage is invoked in the distribution build directory, after watermarking, but before archiving. It can be used to generate distribution time build artefacts. Defaults to massage.
exclude_paths () is invoked in the distribution build directory, after massaging, to determine the paths that are excluded from being added to the distribution archive. Defaults to exclude_paths.
uri is an URI pattern that specifies the location of the distribution on the WWW. In this string any sub-string "$(NAME)" is replaced by the package name, "$(VERSION)" is replaced by the distribution version string and "$(VERSION_NUM)" by the distribution version string, chopping an initial 'v' or 'V' character if present. This argument is used to generate the url file of an opam package for the distribution; it will be deprecated in the future in favour of a x-distrib-uri field in the opam file. If the value is unspecified it defaults to:
```
PKG_HOMEPAGE/releases/$(NAME)-$(VERSION_NUM).tbz
```
where PKG_HOMEPAGE is the package's opam file homepage field. As a special case if the hostname of PKG_HOMEPAGE is github the following is used:
```
PKG_DEV_REPO/releases/download/$(VERSION)/$(NAME)-$(VERSION_NUM).tbz
```
where PKG_DEV_REPO is the package's opam file dev-repo field without the .git suffix and a possible git+ prefix.

val watermarks : watermark list

watermarks is the default list of watermarks. It has the following elements:

("NAME", `Name)
("VERSION", `Version)
("VERSION_NUM", `Version_num)
("VCS_COMMIT_ID", `Vcs [`Commit_id])
("PKG_MAINTAINER", `Opam (None, "maintainer", ", "))
("PKG_AUTHORS", `Opam (None, "authors", ", ")
("PKG_HOMEPAGE", `Opam (None, "homepage", " ")
("PKG_ISSUES", `Opam (None, "bug-reports", " ")
("PKG_DOC", `Opam (None, "doc", " "))
("PKG_LICENSE", `Opam (None, "license", ", ")
("PKG_REPO", `Opam (None, "dev-repo", " "))

Prepending to the list overrides default definitions.

val files_to_watermark : unit -> fpath list result: files_to_watermark () is the default list of files to watermark. It is invoked in the distribution build directory and gets the set of tracked files of this directory from which it removes the files that end with .eps, .flv, .gif, .ico, .jpeg, .jpg, .mov, .mp3, .mp4, .otf, .pdf, .png, .ps, .ttf, .woff.

val massage : unit -> unit result: massage is the default distribution massaging function. It is invoked in the distribution build directory and does nothing.

val exclude_paths : unit -> fpath list result

exclude_paths () is the default list of paths to exclude from the distribution archive. It is invoked in the distribution build directory and returns the following static set of files.

fun () -> Ok [".git"; ".gitignore"; ".gitattributes"; ".hg"; ".hgignore";
              "build"; "Makefile"; "_build"]

Distribution publication description

type publish: The type for describing distribution publication.

val publish : ?⁠artefacts:[ `Doc | `Distrib | `Alt of string ] list -> unit -> publish

publish ~artefacts () influences the distribution publication process performed by the topkg tool:

artefacts defines which artefacts are published by an invocation of topkg publish without arguments (defaults to [`Doc;`Distrib]).

Package description

type std_file: The type for specifying a standard file.

val std_file : ?⁠install:bool -> fpath -> std_file: std_file ~install p is a standard file p expressed relative to the distribution root directory. The file is automatically installed if install is true (default).

type meta_file: The type for specifying an OCamlfind META file.

val meta_file : ?⁠lint:bool -> ?⁠install:bool -> fpath -> meta_file: meta_file ~lint ~install p is a META file p expressed relative to the distribution root directory. The file is automatically installed in the lib field if install is true (default). If lint is true (default), it is OCamlfind linted.

type opam_file: The type for specifying an opam file.

val opam_file : ?⁠lint:bool -> ?⁠lint_deps_excluding:string list option -> ?⁠install:bool -> fpath -> opam_file

opam_file ~lint ~lint_deps_excluding ~install p is an opam file p expressd relative to the distribution root directory such that:

If install is true (default), it is automatically installed in the lib field.
If lint is true (default), it is opam linted.
If lint_deps_excluding is Some excludes, topkg checks that each of the opam package dependencies is mentioned as a root package in the OCamlbuild _tags file and vice-versa. The following package names are excluded from this test:
- The packages names mentioned in excludes.
- Package names that start with "conf-"
- Topkg_care.OCamlfind.base_packages
- Topkg_care.Opam.ocaml_base_packages
If None the dependency check is disabled.

val describe : ?⁠delegate:Cmd.t -> ?⁠readmes:std_file list -> ?⁠licenses:std_file list -> ?⁠change_logs:std_file list -> ?⁠metas:meta_file list -> ?⁠opams:opam_file list -> ?⁠lint_files:fpath list option -> ?⁠lint_custom:(unit -> R.msg result list) -> ?⁠distrib:distrib -> ?⁠publish:publish -> ?⁠build:build -> string -> (Conf.t -> install list result) -> unit

describe name install describes a package named name with:

delegate, the package delegate command to use. If unspecfied determined by the delegate lookup procedure, see topkg help delegate for more information.
readmes are readme files, defaults to [std_file "README.md"]. Automatic install is in the doc field.
licenses are license files, defaults to [std_file "LICENSE.md"]. Automatic install is in the doc field.
change_logs are change logs, defaults to [std_file "CHANGES.md"]. The first file of the list is the one that is acted upon by the topkg log command. Automatic install is in the doc field.
metas the package's ocamlfind META files, defaults to [ meta_file "pkg/META" ].
opams the package's opam package files, defaults to [opam_file "opam"]. The default opam file used by a package description depends on the package name (which can be overriden from the command line). The opam file lookup procedure selects the first path in opams whose filename is (name ^ ".opam") and, failing to do so, it fallbacks to an "opam" file at the root of the distribution.
lint_files if Some files, ensures that all files mentioned in readme, license, change_log, metas, opams and files are present in the distribution. Defaults to Some []. If None disables the file existence tests (including readme, change_log, license, metas, opams, metas.)
lint_custom defines a custom linting process run with the current directory set at the root of the distribution. Successes and errors in the returned list are reported as such and any error in the list makes the lint fail. Defaults to None.
distrib, specifies the distribution process, defaults to Distribution description ().
publish, specifies the publication process, defaults to publish ().
build, specifies the build process, defaults to Build description ().
install given a build configuration specifies the install moves. Note that some of standard files are automatically installed and don't need to be specified, see std_file, meta_file and opam_file.

Package distribution creation details

The following describes the exact steps performed by topkg distrib to create the distribution archive. Note that topkg allows to override or disable part of the process via command line arguments, e.g. to specify the version string manually or skip linting. See topkg distrib --help for more information.

The distribution process assumes that the source repository working directory is clean so that its definitions are consistent with those of the distribution build directory. A warning is generated if this is not the case as it may end up in inconsistent distribution archives (but which may be fine to only publish a documentation update).

Let $NAME be the name of the package, $BUILD be its build directory, $VERSION be the VCS tag description (e.g. git-describe(1) if you are using git) of the source repository HEAD commit and distrib the distribution description found in the source's repository pkg/pkg.ml file.

Clone the source repository at HEAD as the distribution build directory $BUILD/$NAME-$VERSION.build.
Prepare the distribution:
1. Invoke the files_to_watermark function of distrib in the distribution build directory to determine the files to watermark with watermarks and perform the watermarking process.
2. Adds a version field with value $VERSION to the opam files mentioned by Pkg.describe.
3. Run the massage function of distrib in the distribution build directory. This can be used to create distribution time build artefacts.
Invoke the exclude_paths function of distrib in the distribution build directory to determine the paths to exclude from the archive.
Create a distribution tarball $BUILD/$NAME-$VERSION.tbz with the file hierarchy in $BUILD/$NAME-$VERSION.build, excluding the paths determined at the preceeding point and delete the clone $BUILD/$NAME-$VERSION.build. File modifications times in the archive are set to HEAD's commit time and file permissions are preserved. Any other form of file metadata is discarded in the archive.
Test the distribution. Unpack it in directory $BUILD/$NAME-$VERSION, lint the distribution, build the package in the current build environment with its tests, run the tests, on success delete $BUILD/$NAME-$VERSION. Note that this uses the archive's pkg/pkg.ml file, which should not be different from the source's repository file if the latter was clean when topkg distrib was invoked.

Note on watermarking

It is right to doubt the beauty and be concerned about the watermarking process. However experience shows that alternatives like having an OCaml module generated with the appropriate information doesn't work well in practice. Version numbers do not only show up in OCaml source code. They also appear in documentation comments, metadata files, textual data files and non-OCaml source files.

Watermarking by default all the non binary files of the distribution allows one to write %‌%VERSION%% in any context and be sure it is be substituted with the right version number in dev package (`Pin) and distribution (`Distrib) build contexts (this occurence was not subsituted because a ZERO WIDTH NON-JOINER U+200C was introduced between the first two percent characters).

If this scheme poses a problem for certain files or you remain unconvinced, simply filter the result of files_to_watermark or replace it by the exact files you would like to watermark.