2024 to 2026
Weazly
SEC filings intelligence with an AI research layer
Role: Lead engineer. Owned the Next.js app end to end; majority author of the Rails API.
- weazly.com ↗product site
- lines of code
- ~28klines of code
- scrape interval
- 60sscrape interval
- SEC form categories
- 5SEC form categories
- commits
- 462commits
Context
What this is for
An analyst watching a company has to know when it files, and then has to read the filing. SEC EDGAR gives you both halves badly: the current-filings feed is an Atom document with no cursor, and the filings themselves are decades of inconsistent HTML. Weazly sits between the two.
Problem
The thing that was actually hard to get right
Three problems that look unrelated and are not. First, detect new filings within a minute without re-processing the same ones, against a feed that has no "since" parameter. Second, let a user highlight a passage in a filing and still see that highlight next week, in a document whose markup you do not control. Third, let them ask questions across several filings and get answers whose every claim links back to the sentence it came from.
What I built
The surface area
- Keyword and company subscriptions, scoped per SEC form category, with email alerts fired by a scraper running every sixty seconds.
- A filing taxonomy mapping roughly forty SEC form codes into five user-facing buckets: press releases, financial reports, prospectuses, proxy statements, everything else.
- Full-text filing search against the SEC, a company's complete submission history, and multi-select bulk download zipped in the browser.
- Persistent annotation: highlight any span of a filing, attach a note, and find it there on reload.
- AI research projects: pick companies and specific filings, ask questions, get answers with inline citations that open the exact source document.
- A usage-metered credit system where the per-model rate is a database row rather than a deploy.
The hard parts
Decisions worth defending
Each of these names its trade-off, because a decision without one is a preference.
Building a cursor for a feed that has none
EDGAR's current-filings endpoint returns the latest entries and offers no way to ask for everything since a given point. So the service keeps its own high-water mark: it caches the last accession number it saw, takes entries from the top of the feed while they differ from it, reverses them to oldest-first so ordering is stable, and re-caches the newest. Two HTTP requests per entry, each preceded by a deliberate 125ms sleep to stay at eight requests a second against the SEC's cap of ten.
The trade-off
The whole correctness budget is a 200-entry window, a one-hour cache TTL and a 60-second schedule. In a filing burst those can collide, and the worker runs single-threaded precisely so two passes never overlap. A durable cursor column would be more correct than a cache key, and is the change I would make first.
Highlights that survive a document you do not own
A filing is thousands of nested text nodes with no stable identifiers, so a highlight cannot be anchored to an element. The annotation layer walks the document with a TreeWalker, flattens it into a single character stream, and stores each note as an absolute start and end offset into that stream. On load it walks again, splits text nodes at the boundaries and injects the marks. New selections are checked against existing notes for overlap and merged server-side in one request, and the walker refuses to split across table cells so a highlight cannot break a financial table's structure.
The trade-off
Offsets are only valid while the source document is byte-identical. SEC filings are immutable once filed, which is the single assumption that makes the whole approach safe. It would be the wrong design for anything editable.
Citations that stay clickable after the files are gone
The model returns citations as opaque markers carrying indices into its own response text. Replacing them left-to-right invalidates every later index, so they are applied in descending order of position. Each marker names a file by the id it was uploaded under, and uploaded files are deleted after a run to keep storage clean, while conversation threads are kept so a user can resume. That combination means a resumed thread cites file ids that no longer exist anywhere. The fix is a JSON column on the project that accumulates every file-id-to-source-URL mapping ever seen and merges on both read and write, so a citation written in March still opens the right filing in November.
The trade-off
That map only grows. It is small enough not to matter at this scale, and it is the cheapest way to make a provider's garbage collection invisible to the user.
Ownership
Who wrote what
Commit counts from the repository history, both of my git identities summed. I would rather give you this than a claim you have to take on trust.
| Surface | Commits | My share |
|---|---|---|
| Next.js app | 180 | 97% of commits, owned end to end |
| Rails API | 222 | 69% of commits, majority author |
Stack
What it runs on
Frontend
- Next.js 16
- React 19
- Redux Toolkit
- Stripe
- JSZip
- Motion
Backend
- Rails 8.1
- Ruby 4.0
- PostgreSQL
- Sidekiq + sidekiq-scheduler
- Redis
- Nokogiri + HTTParty
- OpenAI Assistants (file_search)
- Devise + JWT
- Pundit
Infrastructure
- Heroku
- Resend
- Stripe
Testing & quality
- RSpec (37 specs)
- 58% line coverage on the API
Where it is now
Current status
Live at weazly.com. It shipped under time pressure and I can name its defects: the payment webhook logs rather than credits, so crediting leans on the browser completing the return redirect, and the alert fan-out notifies only the first matching user per filing. Both are in the code, both have a one-line shape, and neither has been prioritised against newer work.