Rob's NNCP Guide

---

NNCP, or Node-to-Node Copy, is a suite of store-and-forward programs for Unix-based systems, created in 2017 by Russian developer Sergey Matveev. I first learned about NNCP from an excellent post by John Goerzen^, and I first gave it a try in early 2023. I now use it regularly to move files and execute tasks among my devices.

There are already a few guides for NNCP on Gemini, but NNCP can be a very confusing application to set up and configure correctly. In-depth documentation seems to be lacking, both on Gemini and on the Web: some helpful tips are buried in the nncp-devel mailing list and are not present in any guide I've seen. I want to address some of those shortcomings here.

This post will describe what NNCP is, how it works, and some examples on how to use it. Most of the examples are expanded versions of notes I wrote for myself. This is a very long document, but NNCP has a lot of depth and power, and I want to capture as much of that complexity as I can.

^ Recovering Our Lost Free Will Online: Tools and Techniques That Are Available Now

---

About NNCP

NNCP is a set of CLI programs for copying files, requesting files and executing remote commands between computers. Many tools exist to do such things already, but most work synchronously: both computers must be online simultaneously and connected to each other to send data. NNCP, on the other hand, can work asynchronously.

According to the project's homepage^, NNCP is "intended to help build up small size (dozens of nodes) ad-hoc friend-to-friend (F2F) statically routed darknet delay-tolerant networks for fire-and-forget secure reliable files, file requests, Internet mail and commands transmission." What does this mean?






The project page further states: "All packets are integrity checked, end-to-end encrypted, explicitly authenticated by known participants public keys. Onion encryption is applied to relayed packets. Each node acts both as a client and server, can use push and poll behaviour model. Also there is multicasting areas support."

At its core, NNCP works similarly to FidoNet or SMTP. Data and requests are staged as "packets" in a spool directory. From the spool, nodes can either connect to each other and exchange packets directly (for synchronous transfers), the source node can transfer packets into a directory (for asynchronous transfers via a regular file system) or the source node can dump packets into "bundles" that can be piped into other applications (for asynchronous transfers via tarballs or other sequential transmission methods). Upon transferring, the destination node loads packets into its own spool; they can then be "tossed" to recover the data or request.

NNCP is a spiritual successor to UUCP (Unix-to-Unix Copy), another software suite for transferring files and remote command execution. UUCP was a common method for sending e-mail in the 1980s and '90s and for many years served as the backbone software for Usenet. UUCP was originally designed to send data over modems, only adding support of TCP/IP in later years. NNCP lacks the ability to communicate over modems, but does improve on UUCP in many other ways, including encryption, multicasting and chunking. Many tasks that can be done with UUCP can also be done with NNCP; for example, John Goerzen has a tutorial on how to run a Usenet leaf node using NNCP^^.

NNCP is fully decentralized, free and open source, and runs on any Unix-like system. It does not run on Windows systems; this is an intentional choice by the author. I use NNCP on computers running Debian, Ubuntu, Fedora Linux and Rocky Linux, as well as on Termux for Android.

^ NNCP (HTTP)

^^ Usenet over NNCP

---

Example Workflow

Suppose node Alice trusts nodes Bob, Carol and Dan. One evening Alice decides to perform some tasks: she wants to send a file to Bob and to Dan, she wants to request a file from Carol, and she wants to execute a command for Bob.




A typical workflow for this situation would proceed as follows.









There are many possible workflows that can achieve the same results.

---

Installing

At time of writing, the most recent version of NNCP is 8.11.0, released on 2024-07-23. I currently run 8.10.0 on all of my devices, so this guide will focus on 8.10.0. NNCP is not included in most software repositories, so I installed it from source on my machines.

NNCP is written in Golang, so you need to install Go to compile it. 8.10.0 requires Go 1.20 or newer, while 8.11.0 requires Go 1.22 or newer. One of my computers is stuck on Debian 9, but I could still run a tarball of Go 1.20 downloaded from GitHub. Once Golang is installed, download the 8.10.0 tarball of NNCP from the downloads page^, verify its integrity, extract it, and `cd` to the source directory.

The source ./config file controls a few key components of NNCP, including the default locations for your personal configuration file and NNCP's spool directory. These can be overridden later, but if you know some components will be located in nonstandard places (i.e. when running in Termux), it may be handy to edit ./config now.

Some important default locations are:






Once you are happy with the ./config file, simply run

$ bin/build

to compile the code.

If the build process succeeds, the ./bin directory will contain the following 24 binaries:

hjson-cli
nncp-ack
nncp-bundle
nncp-call
nncp-caller
nncp-cfgdir
nncp-cfgenc
nncp-cfgmin
nncp-cfgnew
nncp-check
nncp-cronexpr
nncp-daemon
nncp-exec
nncp-file
nncp-freq
nncp-hash
nncp-log
nncp-pkt
nncp-reass
nncp-rm
nncp-stat
nncp-toss
nncp-trns
nncp-xfer

Move the compiled binaries to your location of choice, and be sure to add the location to your $PATH if needed. Create the spool directory and log file in their default locations. You may need elevated permissions to perform these operations.

NNCP is now ready to configure.

^ Prepared tarballs (HTTP)

---

Generating a Configuration File

NNCP supports two configuration modes: a single file in HJSON format^, or a directory structure. The default mode is a single HJSON file, and the file is created using a configuration generator tool.

`nncp-cfgnew` is used to create configuration files. This program outputs HJSON to stdout, so you need to direct the output to a file to save it. In a working directory, run the following command, replacing $CONF_FILE with the name of your file.

$ nncp-cfgnew > $CONF_FILE

The default configuration file name is 'nncp.hjson', and its default location is '/usr/local/etc/nncp.hjson' (if you did not edit './config' before building). In this mode, all the data necessary for running NNCP is located in a single file, including all neighbor configuration and your private keys. This makes it easy to reinstall NNCP if needed and manage with (say) version-control tools.

An example default configuration file, with all comments and blank lines removed, is below.

{
  spool: /var/spool/local-nncp
  log: /var/spool/local-nncp/log
  mcd-listen: [".*"]
  mcd-send: {.*: 10}
  self: {
    id: R7B23ZHLN4XPIBFDFRE7UH5S6A7SUE6CQ3TUUHOQXCDNB6ZQCPBQ
    exchpub: RTX6MK2W6MKZEQZP4IUAVUX5CQXONT5EB6Y5I3KCW4B4ABYJ4ULQ
    exchprv: GCOYCD3AJ424FSUXKKQF2AX442NPTYFSCZ747I32SGJL6XF7P4RQ
    signpub: YI2FTFUBYGX2SEVWLRDXULQ2HWD7YAKVCH4LE44UWUAIRISWTVOQ
    signprv: CUZS7I2AYXXLPEYFCUWQEBRZNH4ONUPQRVYOCVRWGDFQFNJ23Z3MENCZS2A4DL5JCK3FYR32FYND3B74AFKRD6FSOOKLKAEIUJLJ2XI
    noiseprv: DMJNUAYLNGQ2QSIAYCKTG73HVV5CGM6TXP5Z2W7Q7Q2Y65P7QMNQ
    noisepub: XGCF7SR352IPKU7NJ3OGR5YOFBA2NS3Y4FOPLLL7UMFSDFXR3NLQ
  }
  neigh: {
    self: {
      id: R7B23ZHLN4XPIBFDFRE7UH5S6A7SUE6CQ3TUUHOQXCDNB6ZQCPBQ
      exchpub: RTX6MK2W6MKZEQZP4IUAVUX5CQXONT5EB6Y5I3KCW4B4ABYJ4ULQ
      signpub: YI2FTFUBYGX2SEVWLRDXULQ2HWD7YAKVCH4LE44UWUAIRISWTVOQ
      noisepub: XGCF7SR352IPKU7NJ3OGR5YOFBA2NS3Y4FOPLLL7UMFSDFXR3NLQ
      exec: {
        sendmail: ["/usr/sbin/sendmail"]
      }
    }
  }
}

NNCP's directory-style configuration translates the structure of HJSON into a file hierarchy: objects become directories, keys become file names, and values become file contents. This makes configuration much easier to deploy, maintain and secure. To use the directory mode, you need to have an existing HJSON-formatted configuration file--use `nncp-cfgnew` to create one if needed. Then run the following command, replacing $CONF_FILE with the name of your file and $CONF_DIR with the name of a(n extant) directory to store the configuration.

$ nncp-cfgdir -cfg $CONF_FILE -dump $CONF_DIR

If the '-cfg' flag is omitted, `nncp-cfgdir` checks the default location for the configuration file and gives an error if the file is not found.

It's possible to translate from directory mode back to single-file mode:

$ nncp-cfgdir -load $CONF_DIR > $CONF_FILE

At build time, NNCP can only be configured to use one configuration mode by default. If you want to use a configuration that is not the default mode or not in the default location, you can specify the configuration in each NNCP command by specifying the '-cfg' flag, or you can set the $NNCPCFG environment variable. When showing example commands, this guide will not include the '-cfg' flag.

I personally like the flexibility that the directory mode provides. I have about a dozen NNCP nodes and share some common configuration information between them all. If I need to modify something on one node and propagate configuration changes to the other nodes, it's far easier to send out files containing a single change using rsync or Syncthing than it is to manually edit a dozen single-file configurations to reflect the change. It also allows me to fine-tune configuration permissions and harden my setup, as I can deploy secrets from my password vaults more easily. The only major downside to directory mode is that it cannot be encrypted at rest (more on this later).

Now that you have a configuration file/directory, you can add a few initial settings.

HJSON, a user interface for JSON (HTTPS)

---

First Configuring

The basic structure of an NNCP configuration is as follows.

(configuration file/directory)
├─ (general options)
├─ notify (mail notifications)
│  ├─ (packet options)
├─ self (private node data)
│  ├─ id
│  ├─ (public and private keys)
├─ neigh (public node data)
│  ├─ self (configuring yourself as a neighbor)
│  │  ├─ id
│  │  ├─ (public keys)
│  │  ├─ exec (command definitions)
│  │  ├─ addrs (where to reach the node)
│  │  ├─ incoming (where inbound files from this neighbor are saved)
│  │  ├─ freq (where the neighbor can access outbound files)
│  │  ├─ calls (scheduled connections to node)
│  │  ├─ (general options)
│  ├─ alice
│  │  ├─ (neighbor options)
│  ├─ bob
│  │  ├─ (neighbor options)
│  ├─ (neighbors)
├─ areas (multicasting data)
│  ├─ area1
│  │  ├─ id
│  │  ├─ (public and private keys)
│  │  ├─ subs (nodes to send area packets)
│  │  ├─ exec (command definitions)
│  │  ├─ (general options)
│  ├─ area2
│  │  ├─ (area options)
│  ├─ (areas)

For the moment, you only need to worry about the .neigh.self section of this configuration.

Note that "self" appears twice in the configuration: once in a dedicated section holding private data, and again as a neighbor to configure local communication. When generating a new configuration, the "self" neighbor is automatically created. It includes an example `sendmail` command, but it does not specify an incoming or freq directory. This means that by default, your node cannot send or request files to itself--it can only run the example command.

To enable sending files to yourself, add the following line inside the .neigh.self section, replacing $INCOMING_DIR with the location of your inbound-files directory. Paths must be absolute.

incoming: $INCOMING_DIR

If your configuration is using directory mode, create a file called 'incoming' in '$NNCPCFG/neigh/self/' (where $NNCPCFG is the location of your configuration directory). The only contents of 'incoming' are the location of the inbound-files directory.

To enable requesting files from yourself, add the following lines inside the .neigh.self section, replacing $FREQ_DIR with the location of your outbound-files directory. Paths must be absolute.

freq: {
  path: $FREQ_DIR
}

If your configuration is using directory mode, create a directory called 'freq/' in '$NNCPCFG/neigh/self/' (where $NNCPCFG is the location of your configuration directory). Create a file called 'path' in the 'freq/' directory whose only contents are the location of the outbound-files directory.

I configure my NNCP nodes to send and request files to a landing point in the home directory of each device. For the "self" node, the locations I use for inbound and outbound files are, respectively:

/home/$USER/nncp/self/in/
/home/$USER/nncp/self/out/

In single-file mode, the .neigh section of your configuration should look similar to the following.

neigh: {
  self: {
    id: R7B23ZHLN4XPIBFDFRE7UH5S6A7SUE6CQ3TUUHOQXCDNB6ZQCPBQ
    exchpub: RTX6MK2W6MKZEQZP4IUAVUX5CQXONT5EB6Y5I3KCW4B4ABYJ4ULQ
    signpub: YI2FTFUBYGX2SEVWLRDXULQ2HWD7YAKVCH4LE44UWUAIRISWTVOQ
    noisepub: XGCF7SR352IPKU7NJ3OGR5YOFBA2NS3Y4FOPLLL7UMFSDFXR3NLQ
    incoming: /home/user/nncp/self/in
    freq: {
      path: /home/user/nncp/self/out
    }
    exec: {
      sendmail: ["/usr/sbin/sendmail"]
    }
  }
}

The following directories should exist on your machine:

/home/user/nncp/self/in
/home/user/nncp/self/out
/var/spool/nncp

Let's try it out!

---

First Packet

You are now ready to create a test packet and transfer it to yourself. Your first packet will be a dummy file, and you will send it by transferring it into a directory and reading it back into the spool.

Before you begin, check the current status of the spool. Run:

$ nncp-stat
self

Create a dummy file; it can have any name and contain anything you like. The file does NOT need to be placed in the outbound-files directory specified in your configuration file; it can exist anywhere on the file system, and NNCP only needs permission to read it. For purposes of demonstration, I am using a file called "dummy.bin" that is 4 KiB in size.

To create a packet with the dummy file, use the `nncp-file` command:

$ nncp-file ~/dummy.bin self:
2024-08-01T20:14:35Z Tx dummy.bin 4.5 KiB/4.5 KiB 100% (16 MiB/sec)
2024-08-01T20:14:35Z File ~/dummy.bin (4.3 KiB) is sent to self:dummy.bin


If you want to send the file to a subdirectory in the inbound-files directory, append a relative path (including filename) after the colon:

$ nncp-file ~/dummy.bin self:sub/dir/dummy.bin

Running `nncp-stat` again shows your packet in the spool.

$ nncp-stat
self
	nice:   B | Rx:        0 B,   0 pkts | Tx:    4.5 KiB,   1 pkts

Notice that the packet is slightly larger than the file being sent. In order to send files, both the origin and the destination must have enough free space to hold both the packet, which includes overhead data, and the tossed data as it's being processed. Relays only need enough space to hold the packet in transit.

You need a transfer directory to place the transferred packet. For this guide, we'll make a temporary one. NNCP will give an error if a specified transfer directory does not exist.

$ mkdir /tmp/packets/

Transfer the packet into the directory using the `nncp-xfer` command:

$ nncp-xfer -tx -mkdir -node self /tmp/packets
2024-08-01T20:15:22Z Tx [redacted id] 4.5 KiB/4.5 KiB 100% (0 B/sec)
2024-08-01T20:15:22Z Packet transfer, sent to self: 56NJJCBXTQ7ICVW3YG636SEZA7PUJZYMX2YE7H53DBZDETHIT4A (4.5 KiB)




Confirm the packet is in transit by checking the spool again.

$ nncp-stat
self

To read the packet from the transfer directory back into the spool, use `nncp-xfer` again, but this time check for receiving packets.

$ nncp-xfer -rx -node self /tmp/packets/
2024-08-01T20:15:40Z Rx /tmp/packets/..E7H53DBZDETHIT4A 4.5 KiB/4.5 KiB 100% (0 B/sec)
2024-08-01T20:15:40Z Packet transfer, received from self: /tmp/packets/R7B23ZHLN4XPIBFDFRE7UH5S6A7SUE6CQ3TUUHOQXCDNB6ZQCPBQ/R7B23ZHLN4XPIBFDFRE7UH5S6A7SUE6CQ3TUUHOQXCDNB6ZQCPBQ/56NJJCBXTQ7ICVW3YG636SEZA7PUJZYMX2YE7H53DBZDETHIT4A (4.5 KiB)

Check with `nncp-stat`:

$ nncp-stat
self
	nice:   B | Rx:    4.5 KiB,   1 pkts | Tx:        0 B,   0 pkts

Your dummy file is now queued in the spool, but it hasn't reached its final location on the file system yet. You still need to process ("toss") the packet and recover the original file. The command to do so is `nncp-toss`:

$ nncp-toss
2024-08-01T20:15:48Z Rx file 56NJJCBXTQ7ICVW3..E7H53DBZDETHIT4A 4.0 KiB/4.0 KiB 100% (0 B/sec)
2024-08-01T20:15:48Z Got file dummy.bin (4.0 KiB) from self

One final check of the spool shows that the packet is now gone:

$ nncp-stat
self

Congratulations, you've sent your first packet with NNCP! You can now delete the contents of '/tmp/packets/'.

---

Adding a Neighbor

If all you did with NNCP was send files to yourself, it would be a pretty useless tool. Let's look at how to add a neighbor to your node.

Usually each machine you install NNCP on will have only one node configured for it. However, there's nothing preventing you from running multiple nodes on a single device--and that's what we'll do for testing purposes. Simply run `nncp-cfgnew` again to create a new configuration file with new node keys. From here on, I'll call this new configuration file the "secondary" node, and the original configuration file your "primary" node. Create a new spool directory (for example, /var/spool/2nncp for the secondary node and add its location to the secondary configuration.

You will need to choose names for your nodes. It's best to use characters that don't require escaping, to make things like scripting easier. For this guide, we'll use the above names. Convert the secondary configuration to the same mode as your primary configuration.

If you use single-file mode for your primary configuration, you can get neighbor information from the secondary in two ways. The most obvious method is to open the secondary configuration file in a text editor and copy the .neigh.self section from there.

The second method uses the command `nncp-cfgmin` to output a minimal version of the secondary configuration to stdout. From there, the .neigh.self section can be copied to the primary configuration.

$ nncp-cfgmin -cfg $SECONDARY_NNCPCFG

Once you've copied neighbor information from the secondary to the primary, edit the primary's configuration file. Neighbor names must be unique, so change the neighbor name from "self" to "secondary". Configure incoming and freq directory values similar to how the primary is configured, and create their corresponding directories.

Next, specify an address at which you can connect to the secondary node. This will just be your local host for now. Add the following lines to the .neigh.secondary section:

addrs: {
  lan: 127.0.0.1:6627
}

If you use directory mode, simply copy the neigh/self/ directory from the secondary to the primary. Be sure to rename the directory to neigh/secondary/ when you copy it, so you don't overwrite the self directory in your primary configuration! Create a directory named 'addrs' in the secondary directory, and create a file named 'lan' in neigh/self/addrs/. Its only contents should be "127.0.0.1:6627".

Add the primary node as a neighbor to the secondary configuration using the same steps.

If you are using the single-file configuration mode, the .neigh section of your primary configuration file should look similar to this.

neigh: {
  self: {
    id: R7B23ZHLN4XPIBFDFRE7UH5S6A7SUE6CQ3TUUHOQXCDNB6ZQCPBQ
    exchpub: RTX6MK2W6MKZEQZP4IUAVUX5CQXONT5EB6Y5I3KCW4B4ABYJ4ULQ
    signpub: YI2FTFUBYGX2SEVWLRDXULQ2HWD7YAKVCH4LE44UWUAIRISWTVOQ
    noisepub: XGCF7SR352IPKU7NJ3OGR5YOFBA2NS3Y4FOPLLL7UMFSDFXR3NLQ
    incoming: /home/user/nncp/self/in
    exec: {
      sendmail: ["/usr/sbin/sendmail"]
    }
    freq: {
      path: /home/user/nncp/self/out
    }
  }
  secondary: {
    id: IXTVOWOLQ73QU5OJ24MKCNPLL7GRNIXR3UFIO7G2OO2LLJBP4MZA
    exchpub: TN5D74YCVY44CU3HGEEBP4YXSMO753DOWRHE24DXRDRABPIVXRTA
    signpub: 4IX3LEIFI47SXCOIZZBCEW5MAH6KAIIQWBVDWCI3CBMJRDQA2LAA
    noisepub: S72GC3TVPLAXSL2ODSERI2MG3LVY5EJHVSHW5NWAZ32HN6XNJQMA
    incoming: /home/user/nncp/secondary/in
    freq: {
      path: /home/user/nncp/secondary/out
    }
    addrs: {
      lan: 127.0.0.1:6627
    }
  }
}

The .neigh section of your secondary configuration file should look similar to this.

neigh: {
  self: {
    id: IXTVOWOLQ73QU5OJ24MKCNPLL7GRNIXR3UFIO7G2OO2LLJBP4MZA
    exchpub: TN5D74YCVY44CU3HGEEBP4YXSMO753DOWRHE24DXRDRABPIVXRTA
    signpub: 4IX3LEIFI47SXCOIZZBCEW5MAH6KAIIQWBVDWCI3CBMJRDQA2LAA
    noisepub: S72GC3TVPLAXSL2ODSERI2MG3LVY5EJHVSHW5NWAZ32HN6XNJQMA
    incoming: /home/user/nncp2/self/in
    exec: {
      sendmail: ["/usr/sbin/sendmail"]
    }
    freq: {
      path: /home/user/nncp2/self/out
    }
  }
  primary: {
    id: R7B23ZHLN4XPIBFDFRE7UH5S6A7SUE6CQ3TUUHOQXCDNB6ZQCPBQ
    exchpub: RTX6MK2W6MKZEQZP4IUAVUX5CQXONT5EB6Y5I3KCW4B4ABYJ4ULQ
    signpub: YI2FTFUBYGX2SEVWLRDXULQ2HWD7YAKVCH4LE44UWUAIRISWTVOQ
    noisepub: XGCF7SR352IPKU7NJ3OGR5YOFBA2NS3Y4FOPLLL7UMFSDFXR3NLQ
    incoming: /home/user/nncp2/primary/in
    freq: {
      path: /home/user/nncp2/primary/out
    }
    addrs: {
      lan: 127.0.0.1:6627
    }
  }
}

The following new directories should exist on your machine:

/home/user/nncp/secondary/in
/home/user/nncp/secondary/out
/home/user/nncp2/primary/in
/home/user/nncp2/primary/out
/home/user/nncp2/self/in
/home/user/nncp2/self/out
/var/spool/2nncp

---

TCP Transmission and Bundles

You can now try a second operation: requesting a file synchronously over TCP.

First, create a file to request. As an example, I've created another 4 KiB file called "dummy2.bin". This file needs to sit somewhere inside the secondary node's outbound-files directory; for ease, put it at /home/user/nncp2/primary/out/dummy2.bin.

Open the port you specified in the primary configuration's .neigh.secondary.addrs section in your firewall.

Launch a second terminal and run the following command, replacing $SECONDARY_NNCPCFG with the location of your secondary configuration.

$ nncp-daemon -cfg $SECONDARY_NNCPCFG -bind [::]:6627



With the daemon running, your primary node can request a file from the secondary node. Return to the first terminal and run:

$ nncp-freq secondary:dummy2.bin

Call the secondary daemon with the following command:

$ nncp-call secondary
2024-08-02T03:57:48Z We have got for secondary: 1 packets, 490 B
2024-08-02T03:57:48Z Connection to secondary (127.0.0.1:6627)
2024-08-02T03:57:48Z Tx JCUI42ELYBO4EKJC..PMFBKYI5JFDA6RKA 490 B/490 B 100% (0 B/sec)
2024-08-02T03:57:48Z Packet JCUI42ELYBO4EKJCZ4SKICMYQK4W5MIPOI5LPMFBKYI5JFDA6RKA is sent
2024-08-02T03:57:59Z Finished call with secondary (0:0:11): 32 KiB received (32 KiB/sec), 33 KiB transferred (33 KiB/sec)

The terminal running `nncp-daemon` outputs the following:

2024-08-02T03:57:48Z primary has got for us: 1 packets, 490 B
2024-08-02T03:57:48Z Connection with primary (127.0.0.1:43162)
2024-08-02T03:57:48Z Rx JCUI42ELYBO4EKJC..PMFBKYI5JFDA6RKA 490 B/490 B 100% (0 B/sec)
2024-08-02T03:57:48Z Got packet JCUI42ELYBO4EKJCZ4SKICMYQK4W5MIPOI5LPMFBKYI5JFDA6RKA 100% (490 B / 490 B): done
2024-08-02T03:57:59Z Finished call with primary (0:0:11): 33 KiB received (33 KiB/sec), 32 KiB transferred (32 KiB/sec)

After 10 seconds of inactivity, the connection closes automatically. The packet is now in the secondary spool's receive queue.

You might wonder why you didn't use `nncp-daemon` to transfer your first test packet to yourself. The reason is that NNCP locks the spool when performing any operations on it in order to prevent conflicts. Running `nncp-daemon` also locks the spool, which prevents you from running `nncp-call` against the spool to connect to the daemon. Using an asynchronous transfer method like `nncp-xfer` allows you to lock the spool sequentially: first to take the packet out, and second to put the packet back in.

The secondary node needs to process the packet and add the requested file to its spool's transfer queue. You use `nncp-toss` to do so:

$ nncp-toss -cfg $SECONDARY_NNCPCFG
2024-08-02T03:58:15Z Tx dummy2.bin 4.5 KiB/4.5 KiB 100% (14 MiB/sec)
2024-08-02T03:58:15Z File /home/user/nncp2/primary/out/dummy2.bin (4.3 KiB) is sent to primary:dummy2.bin
2024-08-02T03:58:15Z Got file request dummy2.bin to primary

At this point the primary node could call the secondary again to receive the file. Suppose, however, that the primary suddenly disconnects and needs to switch to an asynchronous transfer method. The secondary can dump the file packet into a bundle and pipe it into tools that can assist in the transfer. The tool to perform this action is our third packet transmission command, `nncp-bundle`.

`nncp-bundle` and `nncp-xfer` are both used to transfer packets asynchronously, but their methods vary: the latter is used to translate packets into files and directories, while the former is used to translate packets into a data stream. This allows NNCP to easily pipe packets into any tool that processes stdin--or simply provide a convenient way to dump packets into a single file.

Let's send the file you requested from the secondary back to the primary node using `nncp-bundle`, piping it into `gzip` to compress it first.

$ nncp-bundle -cfg $SECONDARY_NNCPCFG -tx -delete primary | gzip > /tmp/packets/response-packet.gz
2024-08-02T04:10:41Z Tx TTZG6HRTJOEWOWZ7..GSL24SA2SS6GNS7A 4.5 KiB/4.5 KiB 100% (0 B/sec)
2024-08-02T04:10:41Z Bundle transfer, sent to node primary TTZG6HRTJOEWOWZ7XYHCQXMBDSPMQEH4OGK7GSL24SA2SS6GNS7A (4.5 KiB)



NNCP is able to check the integrity of the packets it writes and reads. One important property of `nncp-bundle` is that it does not do this check by default.

Ingest the packet on the primary node:

$ gunzip -c /tmp/packets/response-packet.gz | nncp-bundle -rx -check
2024-08-02T04:11:58Z check TTZG6HRTJOEWOWZ7..GSL24SA2SS6GNS7A 4.3 KiB/4.5 KiB 96% (0 B/sec)
2024-08-02T04:11:58Z Bundle transfer, received from IXTVOWOLQ73QU5OJ24MKCNPLL7GRNIXR3UFIO7G2OO2LLJBP4MZA TTZG6HRTJOEWOWZ7XYHCQXMBDSPMQEH4OGK7GSL24SA2SS6GNS7A (4.5 KiB)

Finally, toss the packet:

$ nncp-toss -seen
2024-08-02T04:12:06Z Rx file TTZG6HRTJOEWOWZ7..GSL24SA2SS6GNS7A 4.0 KiB/4.0 KiB 100% (0 B/sec)
2024-08-02T014:12:06Z Got file dummy2.bin (4.0 KiB) from secondary

NNCP has successfully transferred between two nodes.

^ The Noise Protocol Framework (HTTPS)

^^ XDR: External Data Representation Standard (HTTPS)

---

Niceness

Niceness encodes the priority of a packet. Each packet can have a niceness level between 1 and 255 inclusive, 1 being the highest priority and 255 being the lowest. Niceness can be set as an integer directly, as one of four single-letter aliases (optionally plus or minus an integer), or as one of five multi-letter strings.

The four single-letter niceness aliases correspond to four of the five niceness strings:





The fifth string is:


Aliases plus or minus an integer take the following ranges:





---

Acknowledging Packets

After sending a packet with NNCP, you may want confirmation that the recipient received was able to toss it successfully. This can be especially important when sending packets asynchronously. Until you get such a confirmation, you will likely want to keep a copy of the packet in your spool, in case you need to send it again. NNCP provides such confirmation functionality through "ack" packets.

Let's start by sending a file called "dummy3.bin" from the primary node to the secondary using `nncp-xfer`. We'll use the '-noprogress' flag to suppress progress messages.

$ nncp-file ./dummy3.bin secondary:
2024-08-02T22:58:17Z File dummy3.bin (4.3 KiB) is sent to secondary:dummy3.bin
$ nncp-xfer -noprogress -tx -keep -mkdir -node secondary /tmp/packets
2024-08-02T22:58:32Z Packet transfer, sent to secondary: 23LPTDWGCZSRPDSJV2ZYLJFNQOQHV7OFIMB7WO6AQ3TYDD3WHHEQ (4.5 KiB)
$ nncp-xfer -cfg $SECONDARY_NNCPCFG -noprogress -rx -node primary /tmp/packets
2024-08-02T22:58:54Z Packet transfer, received from primary: /tmp/packets/TTZG6HRTJOEWOWZ7XYHCQXMBDSPMQEH4OGK7GSL24SA2SS6GNS7A/IXTVOWOLQ73QU5OJ24MKCNPLL7GRNIXR3UFIO7G2OO2LLJBP4MZA/23LPTDWGCZSRPDSJV2ZYLJFNQOQHV7OFIMB7WO6AQ3TYDD3WHHEQ (4.5 KiB)

When the secondary tosses the packet, use '-gen-ack' to create an ack packet:

$ nncp-toss -cfg $SECONDARY_NNCPCFG -seen -noprogress -gen-ack 
2024-08-02T22:59:05Z ACK to primary of 23LPTDWGCZSRPDSJV2ZYLJFNQOQHV7OFIMB7WO6AQ3TYDD3WHHEQ is sent
2024-08-02T22:59:05Z Got file dummy3.bin (4.5 KiB) from primary


`nncp-stat` now shows an ack packet in the transfer queue of the spool:

@ nncp-stat -pkt
primary
	tx 5VBM4YU32Z547EFWGA7H4S6N7FUHB6MGAAI7UYXSN2O5BVQXTPHA 480 B (nice: MAX)
	nice: MAX | Rx:        0 B,   0 pkts | Tx:      480 B,   1 pkts
self


Send the ack packet back to the primary:

$ nncp-xfer -cfg $SECONDARY_NNCPCFG -noprogress -tx -mkdir -node primary /tmp/packets
2024-08-02T22:59:30Z Packet transfer, sent to primary: 5VBM4YU32Z547EFWGA7H4S6N7FUHB6MGAAI7UYXSN2O5BVQXTPHA (480 B)
$ nncp-xfer -noprogress -rx -node secondary /tmp/packets
2024-08-02T22:59:58Z Packet transfer, received from secondary: /tmp/packets/IXTVOWOLQ73QU5OJ24MKCNPLL7GRNIXR3UFIO7G2OO2LLJBP4MZA/TTZG6HRTJOEWOWZ7XYHCQXMBDSPMQEH4OGK7GSL24SA2SS6GNS7A/5VBM4YU32Z547EFWGA7H4S6N7FUHB6MGAAI7UYXSN2O5BVQXTPHA (480 B)

At this point the primary node has two packets in its spool:

$ nncp-stat
secondary
	nice:   B | Rx:        0 B,   0 pkts | Tx:     23 KiB,   1 pkts
	nice: MAX | Rx:      480 B,   1 pkts | Tx:        0 B,   0 pkts
self

Toss the ack packet and check the spool:

$ nncp-toss
2024-08-02T23:00:03Z Got ACK packet from secondary of 23LPTDWGCZSRPDSJV2ZYLJFNQOQHV7OFIMB7WO6AQ3TYDD3WHHEQ
$ nncp-stat
secondary
self

NNCP detects that the ack packet is referring to the other packet sitting in the spool. Tossing the ack packet automatically removes the original packet, as it is no longer needed.

Acks are not needed when using `nncp-call` if the packet is not a transit packet, since `nncp-call` performs integrity checks by default.

There is an important quirk to note about acks, and it shows up in the example workflow at the beginning of this guide. There, Alice wants Dan to send an ack. Normally Alice would keep a copy of the packet in her spool, and when Dan's ack reaches her, the packet will automatically be removed from her spool. However, the first step in the route uses `nncp-call`. `nncp-call` does not have an option to keep packets, because if Alice calls Carol again, NNCP has no way to tell if Alice needs to resend the packet. If Alice sends Dan multiple packets, she would have to manually track all of the their IDs in order to know which one Dan's ack is for. The solution involves low-level work with routing, which we will cover in the next section.

---

Routing

I will include sample commands here based on the example workflow at the beginning of this guide. You don't need to run them.

In the example workflow, Alice wants to send a file to Dan, but she can't communicate with Dan directly, either synchronously or asynchronously. She needs to send her packet to Carol, who will then relay the packet to Dan. This can be be done using the '-via' flag.

When creating the file packet, Alice specifies that the packet needs to be relayed via Carol:

[alice]$ nncp-file -via carol $FILE dan:

The '-via' switch actually creates a transit packet out of the original packet, with a different ID than the original. The original packet is encrypted using Dan's keys, and the transit packet is encrypted using Carol's keys.

If the packet was not originally created with the '-via' flag (and is therefore not a transit packet), Alice can make a transit packet manually by using `nncp-trns`. If the packet ID is $PACKET:

[alice]$ nncp-trns -via carol dan:$PACKET

Alice calls Carol to send the packet to her:

[alice]$ nncp-call carol

When Carol tosses the packet, the transit packet is decrypted, leaving the original packet for Dan.

Packets can be routed through multiple nodes. If Alice wants to route Dan's packet through Bob and then Carol, she can run:

[alice]$ nncp-file -via bob,carol $FILE dan:

If Alice knows she will always need to route through Carol to reach Dan, Alice can edit Dan's neighbor configuration to route through Carol by default. More information is in a later section. The '-via' flag overrides the value set in the configuration. '-via -' disables routing entirely.

You can now revisit the problem presented in the previous section. To summarize, Alice wants to send a packet to Dan, and she wants to keep a copy of the packet in her spool for Dan to ack. However, the packet's first hop involves calling Carol, which always removes the packet from Alice's spool. The solution uses the `nncp-trns` command.

Alice first creates a packet for Dan that does not route via Carol. She then uses `nncp-trns` to create the transit packet to route via Carol. This leaves two packets in her spool: one to send to Carol, and one for Dan to ack. Alice then sends the transit packet to Carol by calling, and when Dan's ack reaches her, the original packet is removed by the ack.

---

Areas

NNCP packets can have only one destination. So far we've seen NNCP assign nodes as destinations, so the packets we've made can only be received by one node each. Even a relayed packet technically sets its destination as the first hop in the relay, and only after traversing the relay will the packet's destination be set to the final node.

This one-destination limitation means that if you wanted to send one file to 20 nodes, you would need to create 20 packets. Areas are how NNCP solves this problem. NNCP's configuration can add nodes to an "area" and send a packet to the area. When you process the area packet, NNCP automatically creates the necessary packets to send to nodes in the area.

Suppose Alice wants to create an area called "friends" so she can send packets to Bob, Carol and Dan simultaneously. Alice first generates the area ID and keypair with:

alice$ nncp-cfgnew -area friends

Alice edits her configuration, listing Bob, Carol and Dan as subscribers to the area. She then shares the area ID and keys with the others. Bob, Carol and Dan add the ID and keys, and they each specify a directory for incoming files from the area.

When Alice sends a packet to 'friends', she prepends the destination with 'area:' to differentiate it from a node.

alice$ nncp-file $FILE area:friends:

If Alice runs `nncp-stat`, she will see a single initial packet in the 'self' queue. NNCP copies the outbound packets for the area nodes from this initial packet when Alice tosses it. This is to ensure integrity of all the area packets if Alice's node fails in the middle of packet creation.

Alice now creates the outbound packets by tossing the initial packet.

alice$ nncp-toss

If an area packet is sent to a node that has its on list of subscribers, the node will relay the packet to its subs as well as tossing it. In our example workflow, Alice does not have a direct route to Dan, but if Carol adds Dan as a subscriber to the area on her node, then Carol will create another area packet to relay to Dan when she tosses Alice's area packet.

All recipients of an area packet also get a hash file when they toss the packet. The presence of this hash file tells NNCP that the node has already received the packet, and if it arrives from somewhere else, simply delete it when tossing. If Carol also lists Bob as a subscriber and Bob calls her to get Alice's area packet, then his node will ignore a second copy of the packet from Alice herself. Note that because of the subscriber system, area packets cannot be made to relay using '-via' by default: if Carol does not have Dan as a subscriber, Alice must make a transit packet with `nncp-trns` and send both packets to Carol.

An important note about configuring subscribers: do not add "self" as a sub! NNCP produces area packets to the node's subs and writes hashes before tossing the packet contents. This leads to a race condition where NNCP tries to relay an area packet to itself, sees the hash in your spool, assumes you've already received the packet once, and deletes it from your spool before tossing the contents. The result is that your packet gets eaten. This problem plagued me for days before I finally figured out why none of my files were being delivered.

---

Scheduling Calls

So far, when we've called neighbors to transfer packets, we've always used `nncp-call` to do it. This is a single-run command meant to be invoked actively by a user or script. NNCP also has the related tool `nncp-caller`, which is used as a daemon to carry out scheduled calls to a neighbor.

Each neighbor in your configuration has an optional "calls" key that stores a list of cron-like expressions. These expressions simply tell `nncp-caller` when to call the neighbor. Many options can be used in conjunction with the schedule, such as specifying the connection timeout in seconds, setting a maximum niceness to transfer, or only calling in your spool has outbound packets to transfer.

You can check if a cron expression is valid using the `nncp-cronexpr` command.

I don't use `nncp-caller` myself, so I don't have much to say about it, and the NNCP project site's informational page covers the topic quite well.^ Refer there for a full list of options and an explanation of the cron expressions.

^ Call configuration (HTTP)

---

Yggdrasil

NNCP includes support for Yggdrasil, an overlay network that assigns IPv6 addresses based on public keys. I already use the official Yggdrasil implementation on all of my devices, so I have no need for NNCP's built-in Yggdrasil capabilities and therefore have never used them. For more information, consult the NNCP page about Yggdrasil^.

^ Yggdrasil support (HTTP)

---

Command Glossary

Below is a brief description of all 24 programs included with NNCP 8.10.0. They are roughly grouped together by function, in the same order as on NNCP's homepage.

Commands to manage your configuration:

nncp-cfgnew (HTTP)





nncp-cfgdir (HTTP)


nncp-cfgmin (HTTP)


nncp-cfgenc (HTTP)





Commands to create a packet:

nncp-file (HTTP)





nncp-freq (HTTP)


nncp-exec (HTTP)





nncp-ack (HTTP)



nncp-trns (HTTP)


Commands to transfer packets asynchronously:

nncp-xfer (HTTP)





nncp-bundle (HTTP)




Commands to transfer packets synchronously:

nncp-daemon (HTTP)




nncp-call (HTTP)




nncp-caller (HTTP)



nncp-cronexpr (HTTP)


Commands to process packets:

nncp-toss (HTTP)





nncp-check (HTTP)



nncp-reass (HTTP)



Maintenance commands:

nncp-stat (HTTP)



nncp-log (HTTP)


nncp-rm (HTTP)





nncp-pkt (HTTP)


nncp-hash (HTTP)



Other commands:

hjson-cli (HTTPS)


---

Configuration Deep Dive

This section goes through all the options included in NNCP's configuration.

General options:

NNCP must have a spool directory to store packets and a log to write events to. By default, NNCP uses '/var/spool/nncp/' and '/var/spool/nncp/log'. These are the only general options that must be included.



These can be overridden at runtime using the '-spool' and '-log' flags.

NNCP can set a specific umask to use when creating files.


NNCP shows progress by default on every process that takes time.


Set to true to suppress progress by default. Can be overridden at runtime using '-progress' or '-noprogress'.

HDR files contain header information from packets. Certain filesystems such as ZFS gain performance benefits by having a copy of header information in separate, dedicated files


Set to true to disable creating HDR files.

NNCP's daemon can send multicast announcements in a local network. Instances of `nncp-caller` will listen for announcements and use the source address as the preferred route to the node. Note that this multicasting is NOT the same as area multicasting.


Configures on which interfaces `nncp-caller` should listen for multicast announcements.


Configures on which interfaces `nncp-daemon` should send multicast announcements, and how frequently in seconds. Note that `nncp-caller` will remove its preferred route to the node if it does not receive an announcement for 120 seconds.

Yggdrasil options (inside 'yggdrasil-aliases'):

All options in this section are aliases for Yggdrasil parameters. The aliases can be named however you wish. They are used when invoking `nncp-daemon -yggdrasil` or `nncp-call -yggdrasil` to prevent sensitive information from being passed to the command line. Examples from a freshly-generated sample config are:


Sample alias name for your private key.


Sample alias name for the public key of a peer.


Sample alias name for a peer's endpoint for you to connect to.


Sample alias name for a list of peers to try by default. Note that this references "alice-endpoint", an endpoint defined elsewhere in the configuration.

Notification options:

When NNCP successfully tosses a packet, it can send out notifications via sendmail or a similar mail command.


Sub-options include:


Configures to and from addresses when file packets complete.


Configures to and from addresses when freq packets complete.


Configures to and from addresses when exec packets complete. Contains lists of aliases in the format of $NODE.$EXEC_ALIAS or *.$EXEC_ALIAS; for example, if Alice can send an exec packet on Bob's node to run a command called "logrotate", Bob configures the alias 'alice.logrotate' to send notifications when Alice's exec packet tosses.

Self options:

Your ID and keys, including private keys, are stored here.


Sub-options include:


You need an ID with which to identify yourself. This is mandatory.


The source node uses the destination's public exchange key to encrypt packets for the destination to decrypt upon tossing. This is mandatory.


The source node uses its private signature key to sign the packet, which the destination verifies using the source node's public signature key. This is mandatory.


Used in the Noise protocol, a cryptographic protocol that secures communications during synchronous packet transmission. If you only plan to use NNCP asynchronously, such as on an airgapped machine, you do not need this key.

Neigh options:

A self-referential neighbor called 'self' is needed for various NNCP tasks, such as multicasting and notifications, even if you don't specifically plan to send any packets to yourself.

All neighbors need the following information.





'noisepub' is necessary for using `nncp-call` or `nncp-caller`.

If you want to accept file packets from the neighbor, NNCP needs a full path to place the inbound files when you toss the packets.

*incoming

A neighbor can only send an exec packet if you predefine a command for the neighbor to run.


Each entry in exec is a key value pair, with an alias for the command as a key and a list of command-line arguments as the value.

If you do not have a direct route to the neighbor, you can specify a list of other neighbors to relay outbound packets through by default. Any outbound packets will be relayed through the list in order.


This can be overridden at runtime using the '-via' flag.

You can allow the neighbor to request files.


Sub-options include:


Sets the location from which the neighbor can request files. Must be present.


Splits file into chunks of this size in KiB before sending to the requesting neighbor.


Requested files below this size in KiB will have their file packets padded before transmission. Can be overridden at runtime with the '-minsize' flag.


Files above this size in KiB cannot be requested.

You can automatically generate acks for packets received from this neighbor.


Sub-options include:


Set a niceness level for acks (default is 255).


Pad ack packets to this size in KiB.

If you want to call a neighbor, you must know an address to connect to it with. `nncp-caller` can use multicast announcements, but `nncp-call` cannot.


Aliases for predefined addresses. By default, NNCP will cycle through these aliases until one connects. Can be overridden at runtime using `nncp-call $NODE:$ADDRESS`, or can specify an alias to sue at runtime using `nncp-call $NODE:$ALIAS`.

You can set default call parameters per neighbor.


Maximum number of packets per second to receive. Note that this only controls the number of packets; it does not control the speed of transmission.


Maximum number of packets per second to transfer. Note that this only controls the number of packets; it does not control the speed of transmission.


Closes the connection after this many seconds of inactivity (default is 10).


Sets maximum lifespan of the connection, whether or not all data has finished transferring.

NNCP can configure scheduled calls to the neighbor using `nncp-caller`.


Sub-options include:


Cron expression for when to call. This is the only required option.


Closes the connection after this many seconds of inactivity (default is 10).


Only transfers packets of this niceness or lower.


Tosses inbound packets automatically.


Maximum number of packets per second to receive. Note that this only controls the number of packets; it does not control the speed of transmission.


Maximum number of packets per second to transfer. Note that this only controls the number of packets; it does not control the speed of transmission.


Processes only one direction of packets. Set to 'rx' for inbound or 'tx' for outbound.


Checks if you have any outbound packets for the neighbor first. If you don't, this doesn't call the neighbor.


Ignore multicast announcements and forces `nncp-caller` to use predefined addresses instead.

Area options:

Specify multicast areas of neighbors here.


Each area has an alias and a dictionary. Options include:


Configuring this ID is how nodes join the area. Only mandatory option.


Public encryption key for the area. If 'prv' is present but this key is missing, NNCP will give an error.


Private encryption key. If this key is missing, your node can relay area packets but cannot decrypt them.


List of neighbors to relay the area packet to.


Location to place inbound files from members of the area.


Commands that members of the area can create exec packets for. Same format as with 'exec' options for neighbors.


Set to "true" to accept area packets that originate from nodes that are in the area but are not configured as your neighbors. The relayong node must still be someone you trust.

---

As much as this guide covers, there are still some concepts I haven't touched on here, like using '-nock' to skip packet integrity checks and cleaning up your spool with `nncp-rm`. I recommend reading the NNCP project site for further information.

Hope to see you as a neighbor soon!

[This post was originally written on 2024-08-07.]

---

Up One Level

Home

[Last updated: 2024-10-06]

Source