One-day build - Command Line Feed Manager

I develop a command line tool that I can use to manage my RSS/Atom feeds.

Motivation for a Command Line Feed Manager

I want to encourage myself to read more diligently and part of that involves summarizing what I read. The problem with using a tablet, as I have been, is that it requires a huge context switch to start writing notes. The barrier to write is so high that I often just don’t do it.

My new strategy is to do my reading closer to where I do my writing. Since I write on the computer, I should read on the computer. On my tablet, I had a feed reader, but I don’t have one yet on the computer.

I looked through a selection this morning and they all look so bloated. In fact, feed reader is far beyond what I need. I’ve already set up Firefox with all the creature comforts and privacy settings that I want. Why would I give that up to use some chromium-based bloatware?

Instead, I really just want something to track what I should read and what I have read. A feed umm… manager?

Ulterior Motivation: One-day Builds

I like the one-day build model used on Adam Savage’s Tested channel. For one thing, it gives you room to experiment without getting too tied to the final product. But more specifically, it forces you to turn an idea into a plan into action in a short period of time. That’s the kind of pressure that makes diamonds. I wouldn’t want to work like that every day, but I’m hoping that it helps me develop more effective planning strategies.

Requirements

These are the features of a feed reader that I rely on:

1. Check a site and list recent articles.
2. Distinguish between articles that I have read and have not read.
3. Accumulate a list unread articles until I read or remove them.
4. Enable me to apply my own categories to each feed.

That’s not a lot. It’s mostly just very simple querying. With no GUI compoenent, it looks like something I could knock out pretty quickly. Even better, as a CLI app, it would integrate well with my usual workflow which is split between Firefox and VSCodium.

I start to envision the workflow that I want in shell pseudocode:

`feed <filters> | xdg-open` 

No. Not feed. Let’s go with nom. “Nom” is the cartoonish sound of eating. nom keys slightly better than feed and surely the feed-reading namespace is more saturated with “feed” than “nom”.

Let’s make room for two more tweaks. First, an aspect of CLI app design. Running nom with no subcommands should display help. So we need a subcommand to show urls. Second, it turns out that xdg-open operates on a single file or URL, not a list. With that, my envisioned workflow to open entries is:

nom show <filters> | xargs -n1 xdg-open 

That’s our north star.

Designing the Interface

To map out a design, I used markdown with markmap to visualize it.

You can see the whole plan here and the markdown file used to create it here.

Summarizing the design plan:

• Some commands act on feeds and start with nom feed.
• Some commands act on entries and start with nom entry.
• Use freedesktop.org locations for file locations.
• Use a UNIXy design so that we can use grep to filter rows and cut to filter columns. (Eventually built-in filters would be nice.)

It’s really nice how far convention takes us with this design. It wasn’t immediately obvious to separate “feed actions” from “entry actions”, but otherwise many options seemed pretty straightforward when you subscribe a UNIXy design.

Prioritizing Today’s Work

Remember, our north start is nom show <filters> | xargs -n1 xdg-open. We can inject cut and grep in their to take the place of <filters> for the time being. With that in mind, we need to get everything in place for the very basics of nom show to work. (I should mention that nom show is an alias for nom entry show, but since I expect that to be the most common nom command, we might as well make it easy to use.)

Let’s work backward from our demo to see what work we need to do.

• nom show prints a list of (unread) entries.
• (Unread entries are entries found that have not been marked as read.)
• Entries come from reading one or more feeds.
• The feeds are enumerated in a feed list.

Okay, that enumerates all the dependencies of nom show. Now, let’s build out the details by starting from the beginning and filling in each step until nom show prints, at minimum, the URLs that we want to open. Here, I try to be precise and cut out extraneous work.

1. Manually write a feed list to ~/.local/share/nom/feedlist/default.
2. Read the feed list. For each feed in the feed list:
   1. Fetch each feed from the URL.
   2. Compute the filename from the feed id.
   3. Save feed to ~/.cache/nom/feeds/<filename>.xml.
3. Read a feed; append to a list of entries. The list of entries can be a CSV. Save the list of entries to ~/.local/share/nom/entrylist/default/entries.csv.
   1. For each feed in feedlist:
      1. Determine whether the feed is Atom or RSS
      2. For each entry in feed:
         1. Get title, url, (id, published, updated, summary)
         2. Print a line to list of entries
4. Hook functions to CLI.
   1. nom feed update reads default feedlist; (re)populates feed cache; builds the entry list.
   2. nom entry show print the list of entries.

I think that will get all the way there. There are a few details that are worth explaining. They might be based on faulty assumptions, so it’s worth recording here so we can reassess those assumptions later.

1. I can anticipate using multiple feed lists. I don’t want to implement that now, so the ~/.local/share/nom/feedlist directory will have a single, but that leaves us room in the future to add another file and a --feedlist option. I’m okay with this kind of “future-proofing” because it doesn’t add any complexity to what I’m trying to do today.
2. For now, we are deferring the “viewed” aspect of entries. But that’s a feature that I know I want and it’s going to be tricky to implement. You can’t simply remove entries from the entry list after you’ve read it. If you do that, you’ll add them back whenever you rebuild the entry list. So you must keep track of which entries you’ve viewed outside of the entry list. That’s why each entry list gets its own directory even though entry lists are one-to-one with feed lists.
3. Feeds themselves go in .cache because it doesn’t matter if you throw them out or not. You can always refetch them. On the other hand, your feed list defines your feed. If you throw that out, you throw out information that is hard to recreate. Similarly, if you throw out you entry lists, you’ll lose track of any other information stored alongside them (such as which entries you’ve read and which you haven’t.)

Getting Started

Make a Feed List

Create the directory:

mkdir -p ~/.local/share/nom/feedlist"

In an editor, add the following to ~/.local/share/nom/feedlist/default:

https://simonwillison.net/atom/everything/
https://jvns.ca/atom.xml

Maybe someday, I’ll add support for a name or something, but line-delimited URLs is good enough for now.

Start Coding

I guess at this point, I need to pick a language and do stuff. I’m most familiar with Python and it seems to fit okay. Since I’m treating this as a one-day build, I should probably use what’s familiar and has a lot of libraries, so Python it is.

It works!

See the appendix below for the code. This section walks you through the use case. (I need to start doing the GIF recordings, but that’s for another time.)

Okay, so we have this feed list:

cat ~/.local/share/nom/feedlist/default

https://simonwillison.net/atom/everything/
https://jvns.ca/atom.xml

Let’s use that feed list, which is hardcoded as the default, to download those feeds locally:

/home/kyle/projects/utils/nom/.venv/bin/python /home/kyle/projects/utils/nom/src/main.py feed update

Note that all that mess up until main.py is how I call it from my development environment and would be replace with nom when deployed.

Let’s check that the feeds have been downloaded:

ls -l ~/.cache/nom/feeds/

total 472
-rw-r--r-- 1 kyle kyle 330339 Jan  4 21:05 jvns.ca-atom.xml
-rw-r--r-- 1 kyle kyle 150604 Jan  4 21:05 simonwillison.net-atom-everything-

So now, this command:

/home/kyle/projects/utils/nom/.venv/bin/python /home/kyle/projects/utils/nom/src/main.py entry show | cut -d '|' -f 2 | head -n 2 

Produces this output:

https://simonwillison.net/2025/Jan/4/weeknotes/#atom-everything
https://simonwillison.net/2025/Jan/4/i-live-my-life-a-quarter-century-at-a-time/#atom-everything

When I pipe it into xargs -n1 xdg-open, FireFox opens with two tabs that load those URLs. Sweet!

Conclusion

I’m pleased with how far along this got in a day. I spent most of the day working out the command line use cases, which will help if/when I build out the feature set.

Appendix: Code

Keep in mind, this is a hacky one-day build (including this writeup). It’s not pretty, but it wouldn’t be too bad to clean up.

from dataclasses import dataclass
from pathlib import Path
from typing import Optional
from urllib.parse import urlparse, quote, unquote

import os
import requests
import feedparser


FEED_CACHE=Path.home() / ".cache" / "nom" / "feeds"
FEED_LIST=Path.home() / ".local" / "share" / "nom" / "feedlist" / "default"


class NomError(Exception):
    pass

def url2filename(url: str)->str:
    p = urlparse(url)
    stringified = ''.join([p.netloc, p.path])
    return quote(stringified, safe='',).replace('%2F','-')

def filename2url(url: str)->str:
    p = urlparse(url.replace('-','%2f'))
    return "https://" + unquote(''.join([p.netloc, p.path]))


# TODO: This should probably use Pydantic.
@dataclass 
class Entry:
    title: str
    url: str
    updated: str   # TODO: Make this datetime when I add filters
    # summary: str # TODO: Add this when you feel like stripping HTML

    # TODO: What if there's a pipe in one of the fields? 
    def write_line(self, delimiter: str ='|'):
        return delimiter.join([self.title, self.url, self.updated])


class Feed:

    def __init__(self, url: str):
        d = feedparser.parse(url)
        self.d = d
        self.name = d.feed.title
        self.url = url # how is this different from d.feed.link?
        self.entries : list[Entry] = [
            Entry(
                e.title, 
                e.link, 
                e.updated
                ) for e in d.entries]

    # TODO: Fix this with command line option
    def to_stdout(self, file: Optional[Path]=None):
        for entry in self.entries:
            if entry:
                print(entry.write_line())


class FeedList:

    def __init__(self, file: Path):
        with open(file, 'r') as f:
            urls = f.read().splitlines()
        
        self.name = file.name
        self.urls = urls

    def fetch_feeds(self, save_dir: Path=FEED_CACHE):
        if not os.path.exists(save_dir):
            os.makedirs(save_dir)

        for url in self.urls:
            filename = url2filename(url)
            path = save_dir / filename
            with open(path, 'w') as f:
                # TODO: URL Error Handling
                r = requests.get(url)
                f.write(r.text)
            print(f"{path} updated")

# TODO: Need to append feeds to one another (and save to entrylist)
# TODO: Flesh out CLI.
if __name__ == "__main__":
    
    from argparse import ArgumentParser
    parser = ArgumentParser(description="Nom Script")
    subparsers = parser.add_subparsers(dest='command', help='Sub-command help')

    # Entry subcommand
    entry_parser = subparsers.add_parser('entry', help='Entry related commands')
    entry_subparsers = entry_parser.add_subparsers(dest='entry_command', help='Entry sub-command help')
    entry_show_parser = entry_subparsers.add_parser('show', help='Show entries')

    # Feed subcommand
    feed_parser = subparsers.add_parser('feed', help='Feed related commands')
    feed_subparsers = feed_parser.add_subparsers(dest='feed_command', help='Feed sub-command help')
    feed_update_parser = feed_subparsers.add_parser('update', help='Update feed')

    # Parse Args
    args = parser.parse_args()

    # Direct Logic 
    feedlist=FeedList(FEED_LIST)
    if args.command == "entry" and args.entry_command == "show":
        for url in feedlist.urls:
            feed=Feed(str(FEED_CACHE / url2filename(url)))
            feed.to_stdout()
    elif args.command == "feed" and args.feed_command == "update":
        feedlist.fetch_feeds()
    else:
        raise NomError("That option is not yet supported.")