---
title: Using KBFS as a makeshift maven server
description: A free and secure way to host personal Java libraries and applications
date: 2021-02-25
written: 2021-02-22
tags:
- maven
- project
- java
extra:
  excerpt: In my never-ending hunt for a suitable solution for hosting Java libraries,
    I take a stop to try out Keybase Filesystem (KBFS)
redirect_from:
- /post/g4lk45j3/
- /g4lk45j3/
aliases:
- /blog/2021/02/25/kbfs-maven
- /blog/kbfs-maven
---

As I continue to write more and more Java libraries for personal and public use, I keep finding myself limited by my library hosting solutions. Maven servers are currently my go-to way of storing and organizing all things Java. I have gone through a solid handful of servers over the past few years, here are my comments on each:

 - GitHub Releases
   - No [dependabot](https://dependabot.com/) integration
   - No easy way to get Gradle to load files directly from GitHub
 - [JitPack](https://jitpack.io/)
   - Slow builds
   - No easy way to publish custom artifacts or use custom groups
   - Sometimes unusably long cache policy
 - [Ultralight](@/blog/2020-09-17-Ultralight-writeup.md)
   - Has a file transfer limit
   - Uses my personal API keys to interact with GitHub
   - No way to automate package updates
 - [GitHub Packages](https://github.com/features/packages)
   - Requires users to authenticate even for public assets
   - Has a file transfer limit
   - Uses a separate maven url per project

As a student, I prefer not to do the sensible solution--*spin up an [Artifactory](https://jfrog.com/artifactory/) server*--as that costs money I could be spending on coffee.

## What makes a maven server special?

Really, not much. As outlined in my [previous maven-related post](@/blog/2020-09-17-Ultralight-writeup.md), a maven server is just a simple webserver with a specific directory structure, and some metadata files placed in specific locations.

Let's say we wanted to publish a package with the following attributes:

| Attribute  | Value                  |
| ---------- | ---------------------- |
| GroupID    | `ca.retrylife.example` |
| ArtifactID | `example-artifact`     |
| Version    | `1.0.4`                |

The resulting directory structure would end up looking like:

```
.
└── ca
    └── retrylife
        └── example
            └── example-artifact
                ├── maven-metadata.xml
                ├── maven-metadata.xml.sha1
                └── 1.0.4
                    ├── example-artifact-1.0.4.jar
                    ├── example-artifact-1.0.4.jar.sha1
                    ├── example-artifact-1.0.4.pom
                    └── example-artifact-1.0.4.pom.sha1
```

<div class="center" markdown="1">

*Generated with [tree.nathanfriend.io](https://tree.nathanfriend.io)*

</div>

In this example. I chose to use the `sha1` hashing algorithm, but maven clients support pretty much any algorithm I can think of. 

As you can see, the files are layed out very logically. Packages are organized similarly to how you organize your source code; each artifact is accompanied by a [Project Object Model](https://maven.apache.org/guides/introduction/introduction-to-the-pom.html) describing it, `maven-metadata` files keep track of versioning, and every file also has a hash alongside it.

For reference, the `maven-metadata.xml` in this example would look something like this:

```xml
<?xml version="1.0" encoding="UTF-8"?>
<metadata>
  <groupId>ca.retrylife.example</groupId>
  <artifactId>example-artifact</artifactId>
  <versioning>
    <release>1.0.4</release>
    <latest>1.0.4</latest>
    <versions>
      <version>1.0.4</version>
    </versions>
    <lastUpdated>20210216203206</lastUpdated>
  </versioning>
</metadata>
```

As far as I know, `maven-metadata` is not actually required, but I always include them so that I can make use of [dynamic versions](https://docs.gradle.org/current/userguide/dynamic_versions.html) in Gradle.

## Using a static CDN as a maven server

Since there is nothing special about a maven server aside from its directory structure, anywhere that can host files can become a server. My choice for now is [Keybase](https://keybase.io/)'s [KBFS](https://book.keybase.io/docs/files). KBFS is a pgp-signed file store that allows every user 250GB of free storage. This web filesystem is mounted to the user's device using [FUSE](https://www.kernel.org/doc/html/latest/filesystems/fuse.html) in a similar way to [rclone](https://rclone.org/).

This local mount & sync setup allows me to interact with my `/keybase` mountpoint like any other directory, while having all its contents automatically backed up and published.

### Taking advantage of this

Gradle's [`maven-publish`](https://docs.gradle.org/current/userguide/publishing_maven.html) plugin is designed to publish packages to remote servers, but will also work with local URIs. Simply pointing a [`MavenPublication`](https://docs.gradle.org/current/dsl/org.gradle.api.publish.maven.MavenPublication.html) to `/keybase/public/ewpratten/maven/release` (my directory of choice for now) will automatically generate everything mentioned in the section about file structure above.

My exact configuration for doing this in gradle is as follows ([source](https://github.com/Ewpratten/gradle_scripts/blob/master/keybase_publishing.gradle)):

```groovy
apply plugin: "maven-publish"

// Determine SNAPSHOT vs release
def isRelease = !project.findProperty("version").contains("-SNAPSHOT")
if (!isRelease) {
    println "Detected SNAPSHOT"
}

publishing {
    repositories {
        maven {
            name = "KBFS"
            if (isRelease) {
                url = uri(
                    project.findProperty("kbfs.maven.release") ?: "/keybase/public/ewpratten/maven/release"
                )
            } else {
                url = uri(
                    project.findProperty("kbfs.maven.snapshot") ?: "/keybase/public/ewpratten/maven/snapshot"
                )
            }
        }
    }
}
```

This configuration is a bit fancy as it will separate snapshots from releases, and allow me to completely override the endpoint(s) in my `settings.gradle` file if I choose. A minimal approach would be:

```groovy
apply plugin: "maven-publish"

publishing {
    repositories {
        maven {
            name = "KBFS"
            url = uri("/keybase/public/<your username>/maven")
        }
    }
}
```

### Pretty URLs

With the solution outlined in this post, the end user would end up specifying one of the following URLs in their maven client:

 - `https://<username>.keybase.pub/maven/release/`
 - `https://<username>.keybase.pub/maven/snapshot/`

While that is perfectly fine, I prefer to keep all of my projects / services / etc under my personal domain (`retrylife.ca`). Unlike the rest of this post, this step does cost some money.

I already rent two servers for various other projects, and one of them is running the [Caddy](https://caddyserver.com/) webserver and acting as a reverse proxy. I have pointed two domains (`release.maven.retrylife.ca` and `snapshot.maven.retrylife.ca`) at this server and am using the following rules to route them:

```text
release.maven.retrylife.ca {
    route /* {
        rewrite * /maven/release/{path}
        reverse_proxy https://ewpratten.keybase.pub {
            header_up Host ewpratten.keybase.pub
        }
    }
}

snapshot.maven.retrylife.ca {
    route /* {
        rewrite * /maven/snapshot/{path}
        reverse_proxy https://ewpratten.keybase.pub {
            header_up Host ewpratten.keybase.pub
        }
    }
}
```

This means that I can point users at one of the following domains, and they will get the packages they are looking for:

 - `https://release.maven.retrylife.ca/`
 - `https://snapshot.maven.retrylife.ca/`

I am also now able to switch out backend servers / services whenever I want, and users will see no difference.

## Future improvements

Some time in the future, I plan to move from KBFS to the S3-based [DigitalOcean Spaces](https://www.digitalocean.com/products/spaces/) so I can speed up the download time for packages, and have better global distribution of files.