ai-partial

Manages PDFs, images, and text files in a local SQLite catalog with stable node IDs, SHA-256 content addressing, ranked full-text search, tagging, and incremental backup with verification.

Recovers a complete site from a Wayback Machine snapshot – HTML, CSS, JS, images, fonts – rewriting URLs and stripping Wayback artifacts so the result behaves like the original.

Google’s official data export service, producing massive archives covering Gmail, Google Photos, Drive, YouTube history, Calendar, Contacts, Location History, and dozens more services. A single Takeout dump is the largest personal data ingestion event most people will ever perform — and a critical starting point for anyone building a personal digital archive.

A self-hosted, AI-powered photo management application with face recognition, automatic categorization, map views, and album management. In the con/serve architecture, PhotoPrism serves as a rich visualization frontend over git-annex-tracked photo archives, applying the data-visualization separation principle: photos live in git-annex, PhotoPrism provides the browsable UI.

A simple, self-hosted photo gallery designed for photographers who want a clean browsing experience over existing folder structures. Already deployed as a service in Lab-in-a-Box. Lighter than PhotoPrism, it reads photos directly from the filesystem without requiring import or database migration.

A self-hosted internet archiving tool that takes URLs and saves them in multiple formats – HTML, PDF, screenshot, WARC, media files – for long-term preservation.

Flagship tool for archiving YouTube channels and playlists into git-annex repositories with full metadata preservation. Built on yt-dlp with native DataLad integration for incremental, content-addressed video archival.

A single-file Python file server supporting HTTP, WebDAV, SFTP, FTP, and TFTP with resumable uploads, content-based deduplication, media indexing, and a full-featured web interface. Useful as an ingestion front-end and quick file sharing tool within research infrastructure.

hub.datalad.org – a public Forgejo-Aneksajo deployment providing web-based browsing, cloning, and collaboration on DataLad datasets.

Integrates Docker and Singularity/Apptainer container images into DataLad datasets, enabling reproducible computational workflows where both data and execution environments are version-controlled.

A DataLad extension that provides pipeline-based web crawling for systematically tracking and archiving web resources into DataLad datasets.

A fork of Forgejo that adds native git-annex protocol support, enabling self-hosted web browsing, cloning, and collaboration on DataLad datasets. Foundation of the DataLad Hub service.

A fork of Gogs with git-annex support for versioning large research data files, operated by G-Node (German Neuroinformatics Node) at LMU Munich. Funded by the same NSF+BMBF CRCNS program as DataLad.

A mature, well-established website mirroring tool that creates offline-browsable copies of entire websites, preserving directory structure and link integrity.

Python tool for exporting Matrix room messages to structured YAML files, with support for media downloads, E2E encrypted rooms, and SSO authentication.

Universal cloud storage adapter supporting 70+ providers. Serves a dual role in con/serve: ingestion (pull files from cloud) and distribution (push archives to cloud as a git-annex special remote). The universal adapter between the git-annex vault and the cloud storage ecosystem.

A browser extension (and CLI tool) that saves a complete web page – including CSS, images, fonts, and iframes – into a single, self-contained HTML file.

Docker-based Telegram archival tool with incremental backups, media preservation, real-time message capture, and a built-in web viewer.

The Swiss army knife of video downloading. Foundation for many archival workflows, usable standalone with git-annex import or as the engine behind annextube’s DataLad-native integration.