targets and tarchetypes

[wlandau]

Submitting Author: Name (@wlandau)
Repository:

• targets: https://github.com/wlandau/targets
• tarchetypes: https://github.com/wlandau/tarchetypes. (tarchetypes is a small companion package that only serves to extend targets, so thought it appropriate to submit it for review along with targets.)
• The manual: https://github.com/wlandau/targets-manual. (For https://github.com/ropensci-books.)

Version submitted:

• 0.0.0.9002 (targets)
• 0.0.0.9000 (tarchetypes and the manual)

Editor: @maurolepore
Reviewer 1: @limnoliver
Reviewer 2: @tjmahr
Archive: TBD
Version accepted: TBD

---

DESCRIPTION files

• targets

Package: targets
Title: Dynamic Function-Oriented Make-Like Declarative Pipelines for R
Description: The targets package is a pipeline toolkit that brings together
  function-oriented programming and Make-like declarative workflows for
  Statistics and data science in R. It implements a workflow as collection of
  interconnected tasks, analyzes the dependency relationships among these
  tasks, skips steps that are already up to date, runs the necessary
  computations with optional parallel workers, abstracts files as
  R objects, and provides tangible evidence that the results match
  the underlying code and data. The methodology in this package
  borrows from GNU Make by Richard Stallman (2015, ISBN:978-9881443519)
  and drake by Will Landau (2018) <doi:10.21105/joss.00550>.
Version: 0.0.0.9001
License: MIT + file LICENSE
URL: https://wlandau.github.io/targets/, https://github.com/wlandau/targets
BugReports: https://github.com/wlandau/targets/issues
Authors@R: c(
  person(
    given = c("William", "Michael"),
    family = "Landau",
    role = c("aut", "cre"),
    email = "will.landau@gmail.com",
    comment = c(ORCID = "0000-0003-1878-3253")
  ),
  person(
    family = "Eli Lilly and Company",
    role = "cph"
  ),
  person(
    given = c("Matthew", "T."),
    family = "Warkentin",
    role = "ctb"
  ))
Depends:
  R (>= 3.5.0)
Imports:
  callr (>= 3.4.3),
  cli (>= 2.0.2),
  codetools (>= 0.2.16),
  data.table (>= 1.12.8),
  digest (>= 0.6.25),
  igraph (>= 1.2.5),
  R6 (>= 2.4.1),
  rlang (>= 0.4.5),
  tibble (>= 3.0.1),
  tidyselect (>= 1.1.0),
  utils,
  vctrs (>= 0.2.4),
  withr (>= 2.1.2)
Suggests:
  aws.s3 (>= 0.3.21),
  clustermq (>= 0.8.9),
  curl (>= 4.3),
  dplyr (>= 1.0.0),
  fst (>= 0.9.2),
  future (>= 1.17.0),
  keras (>= 2.2.5.0),
  knitr (>= 1.30),
  rmarkdown (>= 2.4),
  qs (>= 0.23.2),
  rstudioapi (>= 0.11),
  testthat (>= 2.3.2),
  torch (>= 0.1.0),
  usethis (>= 1.6.3),
  visNetwork (>= 2.0.9)
Encoding: UTF-8
Language: en-US
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.1.1.9000
VignetteBuilder: knitr

• tarchetypes

Package: tarchetypes
Title: Archetypes for Targets
Description: The targets package is a pipeline toolkit that brings together
  function-oriented programming and Make-like declarative workflows for
  Statistics and data science in R. The tarchetypes package provides
  convenient user-side functions to create specialized targets,
  making pipelines easier to create and read. The methods in this package
  were influenced by the drake R package by Will Landau (2018)
  <doi:10.21105/joss.00550>.
Version: 0.0.0.9000
License: MIT + file LICENSE
URL: https://wlandau.github.io/tarchetypes/, https://github.com/wlandau/tarchetypes
BugReports: https://github.com/wlandau/tarchetypes/issues
Authors@R: c(
  person(
    given = c("William", "Michael"),
    family = "Landau",
    role = c("aut", "cre"),
    email = "will.landau@gmail.com",
    comment = c(ORCID = "0000-0003-1878-3253")
  ),
  person(
    family = "Eli Lilly and Company",
    role = "cph"
  ))
Depends:
  R (>= 3.5.0)
Imports:
  fs (>= 1.4.2),
  rlang (>= 0.4.7),
  targets,
  tidyselect (>= 1.1.0),
  utils,
  vctrs (>= 0.3.4),
  withr (>= 2.1.2)
Suggests:
  digest (>= 0.6.25),
  knitr (>= 1.28),
  rmarkdown (>= 2.1),
  testthat (>= 2.3.2)
Remotes:
  wlandau/targets
Encoding: UTF-8
Language: en-US
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.1.1.9000

• Manual

Package: targets.manual
Title: Targets R Package User Manual
Description: This repository contains the source files of the targets R
  package user manual.
Version: 0.0.0.9000
License: MIT + file LICENSE
URL: https://wlandau.github.io/targets-manual,
  https://github.com/wlandau/targets-manual
BugReports: https://github.com/wlandau/targets-manual/issues
Authors@R: c(
  person(
    given = c("William", "Michael"),
    family = "Landau",
    role = c("aut", "cre"),
    email = "will.landau@gmail.com",
    comment = c(ORCID = "0000-0003-1878-3253")
  ),
  person(
    family = "Eli Lilly and Company",
    role = "cph"
  ))
Depends:
  R (>= 3.5.0)
Imports:
  biglm (>= 0.9.2),
  bookdown (>= 0.19),
  fs (>= 1.4.1),
  purrr (>= 0.3.4),
  tarchetypes,
  targets,
  tidyverse (>= 1.3.0),
  visNetwork (>= 2.0.9),
  withr (>= 2.2.0)
Remotes:
  wlandau/tarchetypes,
  wlandau/targets
Encoding: UTF-8
Language: en-US
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.1.0
VignetteBuilder: knitr

Scope

• Please indicate which category or categories from our package fit policies this package falls under: (Please check an appropriate box below. If you are unsure, we suggest you make a pre-submission inquiry.):

  • data retrieval
  • data extraction
  • data munging
  • data deposition
  • workflow automation
  • version control
  • citation management and bibliometrics
  • scientific software wrappers
  • field and lab reproducibility tools
  • database software bindings
  • geospatial data
  • text analysis

• Explain how and why the package falls under these categories (briefly, 1-2 sentences):

targets is an R-focused pipeline toolkit for Make-like declarative workflows. It resolves the dependency relationships among steps of a data analysis workflow and skips steps that are already up to date.

• Who is the target audience and what are scientific applications of this package?

targets is for R users who maintain computationally intense function-oriented data analysis projects (with large codebases and/or long runtimes). Such projects may include but are not limited to Bayesian statistics, simulation, machine learning, PK/PD, and spatial statistics.

• Are there other R packages that accomplish the same thing? If so, how does yours differ or meet our criteria for best-in-category?

targets is the long-term successor to drake. After four years of development, drake has improved so much that its insurmountable problems have become its most pressing ones. A new package is necessary to advance the capability further. So while I still believe drake is thriving, and even though I will continue to maintain drake indefinitely, I created targets to try to break new ground. At https://wlandau.github.io/targets/articles/need.html#drake, I take a detailed dive into the ways that targets surpasses drake's permanent limitations.

• (If applicable) Does your package comply with our guidance around Ethics, Data Privacy and Human Subjects Research?

N/A

• If you made a pre-submission enquiry, please paste the link to the corresponding issue, forum post, or other discussion, or @tag the editor you contacted.

N/A

Technical checks

Confirm each of the following by checking the box.

• I have read the guide for authors and rOpenSci packaging guide.

This package:

• does not violate the Terms of Service of any service it interacts with.
• has a CRAN and OSI accepted license.
• contains a README with instructions for installing the development version.
• includes documentation with examples for all functions, created with roxygen2.
• contains a vignette with examples of its essential functions and uses. To increase modularity and reduce package check time, all the user-side vignettes actually live at https://github.com/wlandau/targets-manual (deployed to https://wlandau.github.io/targets-manual). Repos at https://github.com/wlandau/targets-minimal, https://github.com/wlandau/targets-stan, and https://github.com/wlandau/targets-keras have the complete source code for example use cases. To avoid encumbering core targets and to avoid maintaining duplicated documentation, the vignettes of the actual package only include the statement of need and design documents. The README is deliberately short and links to all this existing documentation.
• has a test suite. Not all of the tests in targets can be automated, especially when it comes to visualization and profiling, so many of the tests live in non-testthat folders in https://github.com/wlandau/targets/tree/master/tests. Whenever I use a #nocov block, I always include a comment with a justification and/or a reference to one of these semi-automated tests.
• has continuous integration, including reporting of test coverage using services such as Travis CI, Coveralls and/or CodeCov.

Publication options

• Do you intend for this package to go on CRAN?
• Do you intend for this package to go on Bioconductor?
• Do you wish to automatically submit to the Journal of Open Source Software? If so:

JOSS Options

• The package has an obvious research application according to JOSS's definition.
  • The package contains a paper.md matching JOSS's requirements with a high-level description in the package root or in inst/. I have written a paper.md, but I need to run it through my company's scientific disclosure process before I share it. That could take a few weeks. paper.md and paper.bib now disclosed and included inside inst/.
  • The package is deposited in a long-term repository with the DOI:
  • (Do not submit your package separately to JOSS)

• Do you wish to submit an Applications Article about your package to Methods in Ecology and Evolution? If so:

MEE Options

• The package is novel and will be of interest to the broad readership of the journal.
• The manuscript describing the package is no longer than 3000 words.
• You intend to archive the code for the package in a long-term repository which meets the requirements of the journal (see MEE's Policy on Publishing Code)
• (Scope: Do consider MEE's Aims and Scope for your manuscript. We make no guarantee that your manuscript will be within MEE scope.)
• (Although not required, we strongly recommend having a full manuscript prepared when you submit here.)
• (Please do not submit your package separately to Methods in Ecology and Evolution)

Code of conduct

• I agree to abide by rOpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.

[melvidoni]

Hello @wlandau for the submission. Before I address it further, could you elaborate on the need for submitting both packages on a single submission? Could this be split into two different submissions? (Thinking about potential reviewer workload). Please, be thorough.

[wlandau]

tarchetypes is a collection of user-level utilities for targets, and it does not stand on its own. Its purpose is to make targets easier to use, and it requires prior familiarity with the latter. I originally considered implementing both in a single package, and I still view their physical separation as an implementation detail. So I thought that separate submissions would potentially duplicate the overall reviewer-side overhead of learning targets, especially compared to the relatively small size of tarchetypes. In addition, tarchetypes helpers like tar_render() also appear in the manual.

[wlandau]

But I am still happy to split up the submission if you think it best. Please let me know what you decide.

[melvidoni]

Hello @wlandau, thanks for the clarifications. @maurolepore will be your handling editor.

[maurolepore]

@wlandau, thanks for the high quality of this submission. I'm pleased to be the
editor.

Here I first give an overview of what I see, then ask for more information to
better understand if this or another structure is the best for this submission,
and finally comment on the editor checks. Please note the comments preceded by
a bullet point and respond to those preceded by a check-box; you may refer to their
labels.

In this submission I see three packages: targets, tarchetypes, and
targets.manual. I understand that (a) targets supersedes the popular package
drake, also maintained by you; (b) tarchetyes extends targets; and (c)
targets.manual would be published as an rOpenSci book containing the manual
for both targets and tarchetypes. Please correct me if necessary.

With three packages, this submission is more complex than the typical 1-package
submission. This may add extra pressure on resources such as rOpenSci reviewers,
and rOpenSci and CRAN servers. Please help me understand why this is the best
alternative (if possible, point me to the source of information that guided your
decision, e.g. books, articles, conversations with other developers):

• (ml01) How did you decide to supersede drake instead of extend it? Did
  you considered any other option, such as the "edition" approach of testthat 3?

• (ml02) How did you decide that targets and tarchetypes should be two
  separate packages?

• (ml03) Who would you suggest as a reviewer for this submission? Please
  consider people with skills that complement yours, specially if they have
  experience restructuring popular R packages.

Your answers will clarify the fit -- which I leave unchecked.

targets

Editor checks:

• Fit: The package meets criteria for fit and overlap
• Automated tests: Package has a testing suite and is tested via Travis-CI or another CI service.
• License: The package has a CRAN or OSI accepted license
• Repository: The repository link resolves correctly
• Archive (JOSS only, may be post-review): The repository DOI resolves correctly
• Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

Editor comments

README

• (ml04) Under What about
  drake? I see "It [is] has
  become ...". You may remove "is".

• (ml05) Under Help, where it
  says "Post to the [GitHub issue tracker]", the link seems incorrect:
  https://github.com/wlandau/issues.

• (ml06) In general, I see an opportunity to remove duplication and thus make
  the package easier to maintain. For example, the file "README.Rmd" links to
  multiple pages under the same website; a single link to the website might be
  enough and easier to maintain. Similarly, under
  Help I see links to repositories
  other than targets. This may be hard to maintain. In isolation, this may seem
  like a minor issue, but minor issues can compound quickly.

CI

• (ml07) In the file "DESCRIPTION" I see Depends: R (>= 3.5.0), but the
  file ".github/workflows/check.yaml" covers R-release only. Please extend the
  continuous-integration workflow to run R CMD check from R-devel to the R
  version stated in the file "DESCRIPTION"
  (?usethis::use_github_action_check_full()).

tarchetypes

Editor checks:

• Fit: The package meets criteria for fit and overlap
• Automated tests: Package has a testing suite and is tested via Travis-CI or another CI service.
• License: The package has a CRAN or OSI accepted license
• Repository: The repository link resolves correctly
• Archive (JOSS only, may be post-review): The repository DOI resolves correctly
• Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

Editor comments

• (ml08) Same as ml06.

• (ml09) Same as ml07.

targets.manual

Editor checks:

• Fit: The package meets criteria for fit and overlap
• Automated tests: Package has a testing suite and is tested via Travis-CI or another CI service.
• License: The package has a CRAN or OSI accepted license
• Repository: The repository link resolves correctly
• Archive (JOSS only, may be post-review): The repository DOI resolves correctly
• Version (JOSS only, may be post-review): Does the release version given match the GitHub release (v1.0.0)?

Editor comments

• (ml10) The name of the package and GitHub repository don't match: "targets.manual" (with ".") and "target-manual" (with "-"). Because the symbol "-" is invalid in the name of an R package, consider renaming the repository to "targets.manual". The inconsistency might confuse some users.

• (ml11) devtools::install_github() failed with upgrade = "default" and passed with upgrade = "never"`. The error seems unrelated to the package but please check.

devtools::install_github("wlandau/targets-manual", dependencies = TRUE, 
    build_vignettes = TRUE)
#> Using github PAT from envvar GITHUB_PAT
#> Downloading GitHub repo wlandau/targets-manual@HEAD
#> readr (1.3.1 -> 1.4.0) [CRAN]
#> Installing 1 packages: readr
#> Installing package into '/home/mauro/R/x86_64-pc-linux-gnu-library/4.0'
#> (as 'lib' is unspecified)
#> Error: Failed to install 'targets.manual' from GitHub:
#>   (converted from warning) installation of package 'readr' had non-zero exit status

• (ml12) R CMD check found 1 ERROR. Please fix the error or explain if R CMD check is not applicable to this type of package.

#> > checking package dependencies ... ERROR
#>   VignetteBuilder package not declared: ‘knitr’

• (ml13) Tests seem not applicable to the book manual. I see no test in other
  books, e.g. https://github.com/ropensci-books/http-testing and
  https://github.com/ropensci-books/taxize.

• (ml14) I see a mismatch between the copy right holder in DESCRIPTION and
  LICENSE (but the copy right holder in DESCRIPTION and LICENSE.md do match). Please ensure they match.

• (ml15) Please update the words list.

devtools::spell_check()
#>   WORD        FOUND IN
#> ’s        index.Rmd:55
#> indivdual   index.Rmd:91
#> learnings   index.Rmd:55
#> repo        README.md:5
#> toolkits    index.Rmd:45,47,51
#> zenodo      README.md:3

---

Reviewers:

Reviewer: @limnoliver
Reviewer: @tjmahr
Due date: 2020-12-19*

* This is four weeks from the day a second reviewer confirmed. I discussed with the chief editor @melvidoni to allow one week more than usual, because of COVID and because this submission includes 2 packages.

[wlandau]

Thank you for your insightful feedback, @maurolepore. Below, I address ml04-ml15. (I will respond to ml01-ml03 in a separate post.)

• ml04: odd, I thought I removed that grammatical typo earlier. Fixed in wlandau/targets@454ddf5.
• ml05: link fixed in wlandau/targets@a3491b1.
• ml06: I agree, and avoiding duplication is one of my major documentation goals for targets. Addressed in wlandau/targets@a3491b1.
• ml07: I added new CI checks under R 3.5 and 3.6 in wlandau/targets@b1a3261
• ml08: I deduplicated some links in wlandau/tarchetypes@d2915e9. I prefer to keep the existing links to individual package functions.
• ml09: addressed in wlandau/tarchetypes@c2e94ea.
• ml10-ml13: I intend targets-manual to just be a bookdown manual, not an installable package. The only purpose of the DESCRIPTION was to make CI easier. To reduce confusion, in ropensci-books/targets@a2092f9 and ropensci-books/targets@5a69d0c, I removed the DESCRIPTION and moved the relevant content into README.md and packages.R.
• ml14: also fixed in ropensci-books/targets@a2092f9 and ropensci-books/targets@5a69d0c. DESCRIPTION and LICENSE are gone, and README.md is consistent with LICENSE.md.
• ml15: without a DESCRIPTION, devtools::spell_check() no longer works. However, I did fix a spelling issue in ropensci-books/targets@7ed0388 ("indivdual" => "individual").

[maurolepore]

Thanks @wlandau for addressing my comments.

• ml10-13 and ml15:

The only purpose of the DESCRIPTION was to make CI easier. To reduce confusion, in ropensci-books/targets@a2092f9 and ropensci-books/targets@5a69d0c, I removed the DESCRIPTION and moved the relevant content into README.md and packages.R.

• (ml16) I support this (more on ml18 below). Now that I understand it, I prefer your original take -- it aligns with the idea of "research compendium". Please revert to it, then explain the goal, structure, and usage of the repository in the field "Description" of the file DESCRIPTION.

• (ml17) Even if targets.manual is not meant to be built and installed as a package, consider adding the minimum infrastructure to the repository for devtools::check() to pass cleanly -- that seems like the path of least resistance. If this is a bad idea, maybe write a broken test with the error message you want someone like me to read.

--

• (ml18) I support anything that makes your work less error prone and easier to maintain. My comments ml01-ml03 focused on external resources, but I forgot to stress that my main concern is maintainability. Your work is outstanding; the R community will benefit from you investing time in the important things instead of mundane tasks. If those mundane tasks are avoidable (e.g. with continuous integration, an "edition" approach, or rethinking the architecture of you system), then my goal is to ensure you consider those options. Collectively, the rOpenSci community has a lot of experience that should help you make informed, good choices.

[wlandau]

• ml18: Thanks, @maurolepore. This helps me answer your questions. The way I am structuring things now, including the separation into multiple repositories, is largely to support maintainability. I will elaborate in my subsequent posts.

[wlandau]

Below, I answer ml02 and ml03, and I address the physical separation of targets-manual and other documentation. ml01 requires a much deeper answer, so I will address it in its own post.

ml02

I have had more time to reflect since @melvidoni first brought this up, and I will try to elaborate.

I could have easily implemented targets and tarchetypes in the same package. Both are essentially the same system, and tarchetypes is currently very small, supporting only a limited handful of superficial extensions to targets' deeper capabilities. So for the present, the physical separation is a trivial implementation detail, and I do not believe it will exacerbate the burden of software review.

The separation of tarchetypes is about planning for the future. From what I learned maintaining drake and observing use cases, tarchetypes will enhance maintainability, the quality of the infrastructure and user-side freedom for years to come.

Sustainable infrastructure

Interface development incurs additional challenges, code volume, bugs, tests, and documentation. I learned this the hard way while developing static branching in drake. Because drake's design did not allow me to implement static branching in a separate package, drake itself became more difficult to maintain, more prone to feature creep, and more prone to errors (see ropensci/drake#1199, ropensci/drake#1262, ropensci/drake#1010, ropensci/drake#1009, ropensci/drake#1008, and many more issues like these). And although I fixed all the known and reported bugs, the deeper underlying causes only worsened as the years went by, and the fundamental design of the internals and the interface made it impossible to eliminate these problems in drake itself.

As its own package, tarchetypes has plenty of room to explore alternative interfaces and shorthand for targets and pipelines. That way,targets itself will stay light, elegant, and sharply delineated in scope, and the architecture will remain clean, resilient, and sustainable in the long run.

A precedent for extensibility

In addition, tarchetypes deliberately sets a precedent for enhanced automation through third-party interface development. tarchetypes functions like tar_render() and tar_map() are easy to implement using targets::tar_target_raw(), and even now, developers are already borrowing the pattern to develop their own specialized interfaces. Internally at work, for example, one of my colleagues maintains a package for domain-specific simulation studies in the life sciences, and it has custom target archetypes tailored to the specific needs of these internal pipelines. An earlier version of this same package used drake, and the metaprogramming on top of drake_plan() was messy and nearly impossible.

targets-manual

In the early days of drake, all the documentation lived in the README and vignettes. Over time, the vignettes steadily accrued volume, length, and R package dependencies, and drake became difficult to install. So I moved the vignettes to their own bookdown project, and the problem was solved.

With targets-manual, I am trying to do the same thing right from the start, even though it is much shorter than drake's manual at the moment. So targets-manual is just a collection of user-side vignettes that happens to be in its own GitHub repository. As with tarchetypes, I do not believe the physical separation of targets and targets-manual increases the burden of software review relative to unifying them in a single package. Conceptually, they still belong together.

Other documentation

targets has additional repositories with extra documentation. They help people learn the package, but I am not submitting them to rOpenSci.

1. https://github.com/wlandau/targets-minimal
2. https://github.com/wlandau/targets-stan
3. https://github.com/wlandau/targets-keras
4. https://github.com/wlandau/targets-tutorial

(1)-(3) are example projects that use targets, and (4) is a half-day short course. Again, I find the physical separation to be important. For drake, I made the mistake of curating multiple example projects in https://github.com/wlandau/drake-examples, and the repo grew without bound. With different projects in different repos now, everything is easier to maintain. The code is easier to fix, and it is easier to set up RStudio Cloud workspaces to help people learn. In addition, the pattern demonstrates to users that each targets-powered data analysis workflow should live in its own repo, which is exactly how I have observed most successful projects to be implemented in practice.

ml03

My top reviewer recommendations are power users who have already provided extensive feedback and expressed strong enthusiasm about targets. Their prior familiarity could reduce the workload.

• @MilesMcBain
• @noamross
• @kendonB
• @mattwarkentin

Otherwise, it may help to reach out to folks who have spread drake throughout the community. Here are some names that come to mind.

• @krlmlr
• @thebioengineer
• @aedobbyn
• @matt-dray
• @matthewstrasiotto
• @tjmahr
• @sinarueeger
• @cstawitz
• @bpbond
• @gadenbuie
• @billdenney
• @tiernanmartin
• @b-rodrigues
• @DominikRafacz
• @pat-s
• @maelle (editor of drake's software review: drake (R package) #156)
• @karthik (though I am not sure if co-founders also review packages)

[tjmahr]

I have been using targets/tarchetypes for one my projects for the past month. I would be able to provide feedback as a desktop/local machine workflow user.