September was a "bed-in-and-bugfix" month, with most of the attention going to the US datacenter trial, node refreshes, and new bundles.
I'm happy to announce that for the second month, we covered our cash expenses!
To get us started, here are some shiny stats for Sept 2024, followed by a summary of some of the user-facing changes announced this month in the blog...
The stats below illustrate CPU cores used (not percentage). These stats are incomplete this month, since ~20 users have migrated to the 3 US cluster nodes, neither of which are visible on this graph.
Examination of the kubectl top nodes output reflects the transition from elves (contended), and hobbits / rangers (dedicated) to "hansels" (contended) and "gretels" (dedicated), as we've added additional bundle types (halflings and nazguls).
Why Hansel & Gretel?
Bundles are datacenter-agnostic, but nodes are specific to each datacenter, and we needed a way to differentiate US nodes from DE nodes. The fairy tale of Hansel and Gretel originates in Germany
This graph represents memory usage across the entire (DE) cluster. Interestingly, tenant memory usage has is on par with last month, while CPU usage is significantly less.
Other high consumers of RAM:
rook-ceph: uses RAM for caching on its network-storage-related workloads
csi-rclone: used for mounting all rclone-compatible storage mounts, primarily RealDebrid libraries
kube-system: the Kubernetes control plane, including the cilium agents which manage the networking / policy enforcement (currently 11K flows/s across 30 nodes)
traefik: all inbound access to the cluster / services
We perform daily backups of client volumes throughout the day, on the hour, in 24 evenly-distributed batches. As a result, we see spikes of ceph throughput on the hour:
In general, the contended and dedicated nodes show slightly more traffic incoming vs outgoing (video being transcoded, or streamed with buffering), which is a big improvement over previous months where Plex/Aarr configurations showed huge amounts of ingress for relatively little egress, as symlinked video files were "analyzed" but not streamed!
Last month (Aug 2024)'s for comparison:
Retrospective
US cluster
About 20 users have made the switch to the US cluster. We've been through a few weeks of "fine-tuning" for the resource constraints in the US (less RAM/CPU, more bandwidth), and with the Riven RAM database limits we've now applied, the platform has been stable for the past week or so.
Here's what the cluster looks like today (compare this with the kubectl top output in the CPU/RAM sections above):
root@eagle01:~# k top nodes
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
eagle01 475m 5% 10632Mi 33%
yankee01 2327m 29% 20906Mi 65%
yankee02 867m 10% 21798Mi 68%
yankee03 1130m 14% 19992Mi 62%
root@eagle01:~#
Our biggest concern is the US cluster is now the lack of redundancy on the control-plane (there's only one eagle node, currently), and simply a lack of capacity to meet demand.
There's new hardware on back-order for the past 2 weeks, which I'm told will be available to us in the next week or so, bringing us to 3 eagles (HA FTW!), and doubling our workload capacity to 6 yankees.
If you'd like to gauge whether the US cluster would be more suitable for your workloads, visit https://speed.elfhosted.com and perform some comparative tests!
New store theme and layout
As flagged in last month's report, the store has received a refresh, with a new theme and organizational structure. Apps are now sorted by type (chat, backup, streaming,.. ), and activity (sharing, managing, watching..), and Ideogram's 2.0 model has been brought to bear on our product images.
The checkout process is still clunky though, and adds more friction than we'd like. Turning on the "next-gen" block-based checkout gives us "express checkout" with Apple/Google Pay, but it breaks ElfBuckz, and the ability for users to pick their own username on checkout, so further refinement is required here!
If you've already added non-bundled apps (custom domains, storage mounts, etc) to your bundle, you'll need to cancel / re-subscribe these to to take advantage of the discounted pricing. Create an #elf-help ticket if you need a hand with this!
In the Aug 2024 feedback, many of the responses to "how did you find us?" had to do with the initial free credit / trials we offered. While we can't get trials back (they break billing and bundle-hopping), we wanted to lower the "barrier to entry", so we now have 1-day expiring "demos", allowing prospective new users to "kick the tyres" for 24h, for just $1, once-off.
We have demos for every Plex / Emby / Jellyfin combination with Riven or the Aars, and will be adding more options in future for Torbox, AllDebrid, Debridlink, etc.
Nazgul / Halflings arrive
With the standardization of dedicated node types for DE ("Gretels"), we can now finally offer the larger-sized packages users have been asking for, and during September, we introduced Halfling (50% dedicated) and Nazgul (100% dedicated) bundles!
Of note, both Halflings and Nazguls can burst up to 1Gbps, but on Halflings that 1Gbps is fairly shared with any neighbors, whereas on a Nazgul, it's all yours!
Now that we've explained that bundled subscribers get discounts on extra apps, note that September also saw us add the following new apps:
Tunarr
Tunarr - create live TV channels from media on your Plex servers, and more!
Configure your channels, programs, commercials, and settings using the Tunarr web UI.
Access your channels by adding the spoofed Tunarr HDHomerun tuner to Plex, Jellyfin, or Emby. Or utilize the m3u Url with any 3rd party IPTV player app.
Threadfin
Threadfin is a better-but-still-PITA fork of Xteve, letting you bring your IPTV into Plex. It sort-of works, depending on your provider.
West's Blackhole Script
West's scripts include a "blackhole" shell script, which replaces the functions of RDTClient. It's dethroned RDTClient in https://savvyguides.wiki, and while some of its functions are redundant to us (Plex requests, etc), having an alternative to RDTCLient seems sensible. Especially one which can use to integrate the Aars with Torbox.
Blackhole is included in all bundles
You don't have to do anything to get blackhole - it's included in all Hobbit / Ranger / Halfling / Nazgul bundles!
Petio
Petio is a third party companion app available to Plex server owners to allow their users to request, review and discover content. The app is built to appear instantly familiar and intuitive to even the most tech-agnostic users.
Petio will help you manage requests from your users, connect to other third party apps such as Sonarr and Radarr, notify users when content is available and track request progress. Petio also allows users to discover media both on and off your server, quickly and easily find related content and review to leave their opinion for other users.
Petio is an ongoing, forever free, always evolving project currently in alpha prototype stage and now available!
Requestrr
Requestrr is a chatbot used to simplify using services like Sonarr/Radarr/Overseerr/Ombi via the use of chat!
Features:
Ability to request content via Discord using slash commands, buttons and more!
Users can get notified when their requests complete
Sonarr (V2-V4) & Radarr (V2-V5) integration with support for multiple instance via Overseerr (only for 4k/1080p)
Overseerr integration with support for per user permissions/quotas and issue submission
Ombi (V3/V4) integration with support for per user roles/quotas and issue submission
Fully configurable via a web portal
Doplarr
Doplarr is a CLI tool to provide a Discord-based interface to a Radarr / Sonarr instance. You can use it, for example, to allow Discord users to make requests, without providing direct access to Radarr / Sonarr.
Immich
Immich is a self-hosted replacement for Google Photos. Detailed instructions and video guides are available here.
Pairdrop
Pairdrop provides local file sharing in your web browser.
Inspired by Apple’s AirDrop.
Fork of Snapdrop
Features
File sharing on your local network that works on all platforms.
A multi-platform AirDrop-like solution that works.
Send images, documents or text via peer-to-peer connection to devices on the same local network.
Internet transfers
Join temporary public rooms to transfer files easily over the Internet.
Web-app
Works on all devices with a modern web-browser.
MediaFlow Proxy
MediaFlow Proxy can be combined with public / ElfHosted MediaFusion to allow users to simultaneously stream from RealDebrid with Stremio/Kodi, from multiple devices on multiple networks, without a RD account ban (like Comet's ProxyStreaming feature).
Coming up
Even mooar apps
Apps currently requested can be found (and submitted!) here
Now that we have a mostly-workable theme for the store, the ElfVengers and I will be moving much of the guides from the docs site (https://elfhosted.com) to the store, since this is more suitable for frequent, collaborative edits.
We'll also be moving the ElfHosted blog to the store, for better integration into our products, member-only content (provided by the new membership plugin!), and automated emailing of new blog posts (you'll have seen a few of these recently).
We're not sure yet exactly where the division should be, but the docs site should remain technical and price-agnostic, whereas the store should present what we offer, commercially, as simply and attractively as possible.
US East Coast DC
Now that we have the US West Coast DC humming, we've started the process of building out an East Coast DC, in Pennsylvania. Please submit your suggestions for what the DNS suffix and node names should be!
Torbox Teamup
Wamy from Torbox is keen to team up with us on a product bundle / offering to support ElfHosted Torbox users. We're still ironing out the details, but rather than trying to use Torbox to replace RD for gigantic-library-building, possible applications are:
Using your private trackers with the Aars for "infinite streaming Plex", while continuing to seed back and maintain your ratio.
Adding a "backup" debrid service for your most critical / popular content, so that you have a fallback in the event of RD rate-limits / changes.
Sourcing non-torrent, premium quality from Torbox's usenet sources.
Got ideas for improvements? Send us an EEP (ElfHosted Enhancement Proposal) here!
How to help
Another effective way to help is to drive traffic / organic discovery, by mentioning ElfHosted in appropriate forums such as Reddit's r/plex, r/realdebrid, and r/StremioAddons, which is where much of our target audience is to be found!
I am new here, but today I learned realized that Elfhosted is one of the best free and open source software communities I've seen, and FOSS communities have been at the center of my life since the 90s (Perl, PHP, Symfony, Drupal, Ethereum, etc.). Great open software built by great people who care = great community, and that is something special.
You've done an amazing job @Funky Penguelf with the platform you provide and this place has an awesome mix of active community caretakers and software creators that I've seen here so far like BSM, Spoked, LayeZee and other elf vengers. Keep up the energy, productivity and community and take time to enjoy it and appreciate each other!
I self host and share a fully automated ‘arr stack with Plex. Been doing so for around 4 years. Also recently got into real debrid and hosting a Comet and Annatar for Stremio. The amount of time and head banging I’ve put into it is in the hundreds to thousands of hours. From setting it up to keeping it running smoothly. Let’s not forget the cost of my server and how much it cost to keep it running.
Anyway I wanted to see what ElfHosted was about to compare. Yeah I had the whole thing setup in just a few hours. It also passes the headache of maintaining it to ElfHosted. Will I keep it no because nerdy things and maintaining my server are my hobby and quirky passion project. Will I recommend it to my friends who don’t have the money up front to buy a server, the knowledge to maintain it or desire.
Just my server alone was $2k. Power cost to keep it on yearly is $250ish, annual memberships to RD, Usenet and indexers are around $100. Then whatever a value my free time at. Which is currently at minimum my hourly pay at work or more. Yeah so take the monthly cost of all that and compare to ElfHosted Ultimate Stream package at $39 monthly, add RD to the cost and get nearly all your time back is incredibly cheap.
Lastly it seems like a lot of people forget how quickly an ultimate cable package used to cost. Or how quick paying for every stream service would add up to. Which when using ElfHosted with RD is essentially and more what you get. Quick hint it’s far above the asking price.
As a happy Elfhosted customer—who also self hosts MANY things across about 10 severs (dedicated, VPSes, and VMs running on Synology), I wouldn’t switch to self hosting the services I get from Elfhosted. They just work with very little effort configuring things, and the support the owner and his team provides is second to none. Plus I love being part of a fledgling—but quickly growing—enterprise.
I recently found ElfHosted and decided to start out with the Infinite Starter Kit. Within a week I realized that this was for me and upgraded to the Hobbit plan. Give it another week and I was up to the Ranger plan.
I just love the simplicity and the fact that things just work. For years I've ran a home server and between the constant maintenance and always upgrading harddrives, it became apparent I wanted to make it easier on my self. Enter ElfHosted.
Setup was super easy with the guided documentation and the discord community. It seems that somebody is available at all hours of the day to help with questions. I started with the Aars, which I knew from my prior hosting... but saw a newer product called Riven. I decided to jump in feet first. I enjoy being on the front end of an up and coming replacement for the Aars and will soon be upgrading to the annual plan!
I’ve been using this service for a while now, and honestly, it’s a game-changer compared to anything else I’ve tried for managing my media library. The support is fantastic—super quick, and if the staff aren’t around (which rarely happens), the community steps up right away. I can’t imagine going back to any other platform.
Before this, I had my own setup with a NUC, NAS, and tools like Sonarr and Radarr. It worked pretty well for a while, and my internet speed was high enough to stream without any buffering. But in the end, it wasn’t worth the time or headache of managing all the storage and keeping everything running smoothly.
Now, with this service, everything runs smoothly in 1080p+ with no buffering issues. The interface is really easy to use, which makes managing everything a breeze. Plus, having a whole community of smart people available for guidance is a huge bonus.
I was sold from the start, which is why I quickly upgraded from a 1-month to a 3-month subscription, and I’m planning to switch to a 1-year plan soon. This service totally pays for itself, and I’m sure you won’t be disappointed. It’s been really impressive.
Best possible options for anyone looking for the do-it-all option along with the best customer service ive experienced in this space so far. Id rate it a 6 if I could but its limited to 5/5...
(responding to a Reddit thread re the cost of ElfHosted vs mainstream streaming / self-hosting):
I didn't know that the goal of this project was to compete with large companies running/renting entire DCs. I was under the impression that the goal of this project was to manage the updating of almost selfhosted applications on a shared platform with other users. Basically, be my sysadmin for me.
That being said, paying for services is the 'easy button'. There is a real world cost incurred for the time saved. Time is money. Time is the most valuable currency that exists. Once time is spent, it's forever lost, one cannot retrieve it again (yet). In my mind, there are 3 options for use of time with respect to: mainstream, selfhosting, elfhosted.
mainstream - my time is valuable and I don't want curated content and I don't care what content that I have the ability to consume. I only like what's popular.
elfhosted - my time is valuable, I want my own curated content without being forced to browse past the same damn entry 500 times just to find out that I can't watch the movie I want because it's not available in my current location or was removed last week from mainstream providers.
selfhost - I care about costs and I have nothing but time to waste or I want to learn about the backend of the systems involved. I'll pay for my own VPS/homelab, electricity, manage the OS, manage app updates, figure out how to make the apps talk nice to each other, create my own beautiful frontend.
I know how much my time is worth, does that reddit poster know how much their time is worth? Without knowing what you are worth, you can't make effective capital expenditures with respect to the time it will take to recoup the capital.
I know I don't need elfhosted at all for my use case. I choose to stay with elfhosted because it's my 'easy button'. It's an efficient capital expense for the amount of time it saves me managing my own hardware, apps and saves me electricity costs. I'm also in a situation where I don't have upload bandwidth from my home to serve HD content to myself remotely. If I lived back in a city, I would still be here. My time is worth $$/hr.
"Just wanted to check in here and let @Darth-Penguini and anyone/everyone else know...WOW. I have been struggling with storage for years, maintenance of Docker containers, upkeep, all of it. Elfhosted is so freeing. It's an amazing service that I hope to be a member of for a long, long time!"
"I just have to say, I am an incredibly satisfied customer. I had been collecting my own content for nearly 20 years. Starting off with just a simple external HD before eventually graduating to a seedbox with 100TB of cloud storage attached and fully automated processes with Sonarr and Radarr . However, the time came when the glory days of unlimited Google Drive storage ended. I thought my days of having my full collection at my fingertips via :plex: were behind me, until I found Real-Debrid and ElfHosted.
Now I essentially have the exact same access to content as I had before, but even better. Superior support and community involvement. Content is available almost immediately after being identified. A plethora of tools at my fingertips that give me more control and automation than ever before. Wonderfully well done and impressive! I am looking forward to being a customer for a very long time! Massive kudos to @funkypenguin 🤟
"I would recommend ElfHosted to anyone. It has been great so far and made life a lot easier than running my own setups. If you’re in the fence give them a try and help support this great community."
"thanks for the help and must say this is the best host I every had for my server 🙂 10/10 🙂 All other places I have try have I got a lot buff etc. Your host can even give me full power on a 4K Remux on 200GB big movie file . That's damn awesome 😄"
@tjelite (Discord)
"What an amazing support system these guys have Chris and Layzee i think it was! Both are very patient with me even though I am a newbie at all this. Very thorough and explained everything step by step with me
I couldn’t ask for anything better than the service I have received by these guys! Happy happy client❤️"
"I had my own plex-arrs setup on hetzner for years. Yesterday I deleted everything as elfhosted has gone above and beyond it. And it has a fantastic, active community as well! Very friendly, helpful and like-minded folks always willing to help and improve the system. Top notch!"
"@BSM went above and beyond to make sure I had all the one on one support needed with my sub. Thank you for your patience! Elfhosted continues to be Elftastic !!"
"really enjoying the service from elfhosted. The setup is really easy from the guides on the website. And the help on the discord channel is really quick."