Add node_modules directory to ignored list of files and directories.

[yazeedobaid]

Motivation

In a recent project I was working on, I customized the theme that FSDocs operate on. The customized theme was first processed with NPM, then template files were fed to FSDocs to use. So as a general highlight of the pipeline:

1. run npm install to install node dependencies,
2. run npm run build to generate the final build output from NPM, this includes final template HTML files as well as styling and JavaScript files,
3. run dotnet fsdocs to generate the site and API documentation.

One issue that results from this flow is that we have node_modules directory that resulted from npm install and is needed in npm run build. That directory got processed by FSDocs when it runs. Hence we get in output a node_modules directory with thousands of files! In addition, the run time for FSDocs has doubled.

The workaround I did was to rename the node_modules directory to .node_modules before running step number 3 above. Since FSDocs ignore files and directories that start with a dot. Once FSDocs is done, I rename it back to node_modules.

The issue with this approach is that, during development, we need to run FSDocs and NPM both in watch modes, but we cannot, since node_modules directory has been renamed, hence NPM cannot be run. So we run either of them.

The Change

This PR adds an ignore list-like function that has the node_modules directory and "." criteria. In which it will ignore either of them. Also, I updated the documentation page to highlight this in the Ignored Content section.

[nhirschey]

I don't know node project structure, so my comment might be off base. But is an alternative to use --input to point fsdocs directly at the folder with output from your npm run build command? Something like dotnet fsdocs --input \path\to\fsdocs\content?

If you have a hierarchy like below, dotnet fsdocs --input docs\fsdocs-content will build the site ignoring the node modules content. I don't use node, but I use --input in this way for similar reasons.

\docs
     \npm-project
          \node_modules
     \fsdocs-content
           index-page-made-with-node.html

[yazeedobaid]

It will not be straightforward, since node_modules will reside in the root level of the NPM project, besides the entry point (main.js and pakcages.json files). We can do the structure you proposed but we need to separate the two projects in the docs directory and once NPM is done move output from the build to the FSDocs content directory. So it will be somehow like the current workaround which renames the node_modules directory.

The issue is when you enter development mode, so want to watch both directories then, and on the update of NPM, we need to copy them to FSDocs directory, which is a lot of work. What I'm trying to reach is using the same directory for both, since both operate on HTML files, and node_modules is considered not part of the source root, so it should be ignored by both; NPM and FSDocs.

[dsyme]

Could we use an .fsdocs/config.json or similar? Rather than hard-coding this into the tool

[yazeedobaid]

I like the idea of a config file, either in .fsdocs/config.json or in the root level of the repository fsdcos.config.json. I like the latter more since it will be discoverable more and similar to other tools, like GIT and Fantomas.

I can propose the file to have the following structure for the moment:

{
    "ignoredPaths": ["node_modules/", "docs/md-file-that-is-not-part-of-site.md"]
}

So, the ignored paths array will have paths relative to the root of the project, either directory or specific files. The tool will keep ignoring directories or files that start with a dot, and will check this file for these ignored paths. Is this a good proposal that we can proceed with?

[dsyme]

@yazeedobaid Sounds good! please go with that!