What are you looking for in a host?

Hm, after the initial upload, it shouldn’t really generate much traffic if I can only manage to upload the diff, so it might not be much of an issue for me. I am not yet really familiar with tools like rsync and rclone, and also don’t know how the changes are stored in the Borg repo (e.g. if I move a 1 GB file from one folder to another, does that get picked up as a 1 GB change by the syncing tools?), so I would need to do some more research to see if that would be achievable.

Hetzner also looks nicely priced, but it would’ve been nice if I could choose an even cheaper tier with less storage, as 1 TB is quite overkill for this particular use case. I could of course use it to backup other things.

Not a requirement that it is E2EE, as the Borg repo is already encrypted. Guess my knowledge of these services is biased towards E2EE from previous research for use cases where that was a requirement.

Thanks for the tip, hadn’t hard about Backblaze before. Very reasonable pricing. Would a good strategy then be to schedule rclone to have it synced, or are there other ways that would be better?

My fibre box does TV, phone, and internet all in one. I guess you have one for each? I’m interested to find out if you’ll share. I think asking them what each of them do and understand it is a good first step. Maybe you can get that down to 2 boxes. Good luck!

I’ll try to remember to DM you when/if I get any answers, but I am currently sick and will be traveling soon, so I am not entirely sure when that will be. The boxes in the cabinet aren’t really in the way, so I am fine with the two boxes there. But it would be nice to avoid having to use their WiFi router if possible though. There is in fact a last box that I have packed away in a closet: the decoder for TV signals, which I don’t use since I don’t have the included in my plan (I could discard TV from my plan in exchange from going fro 100 -> 500 MBit/s internet connection, which for me was a no-brainer). So their standard setting is four boxes, and that is only for TV and internet.

Nice! Glad its still working! Definitely triple check with something like https://canyouseeme.org/ when you open ports.

What a great tool! I will definitely make use of this.

I’m a Linux Sys Admin and happy to do my best to help of you have any more questions. At least I’ll try and get you on the right track.

Cheers, I appreciate that. I might just send you a question or two in the future if I am stuck in trying to figure out something, but of course, don’t ever feel obligated to answer.

I 100% agree with you on the rest. Canada isn’t doing anything and at this point I’m ready to give up. I’m not sure where to draw the line anymore and self hosting is a bit of a pain for me these days. Personal life is a bit rough and it’s just so easy to make a gmail account and have them host it.

Yeah, it is difficult to draw the line. With about 15 years of just going with the flow, signing up for and using all kinds of services, I’ve lately (the past two years at least) trying to untangle me from the worst of it. It takes so much time, and every time I learn about something new (e.g. fingerprinting has finally just started creeping me out) I fall down yet another rabbit hole. I am trying to work myself towards as complete control over my data as possible, including an elaborate homelab setup (though that is still some time away). Hence trying to understand this stuff better so I can do it properly.

I hope you get through your stuff in your personal life. This interaction has in any case been greatly appreciated by me.

Oh, before I go, I just realized that the boxes in the cabinet also handles fiber TV signals.

humm, I’ve never seen or heard of this. I’ve only ever been provided one box by my ISP. I have two guesses… Either you can replace your WiFi router with your own and everything will be okay or you’ll have to add a 3rd that is your own and Plug it into the WiFi router and ask them to put it in bridge mode. My guess is they can help you a lot better then me guessing.

From what I’ve understood from previously looking up this with my ISP, is that I connect my own device to the WiFi router they gave me. In that case I have four boxes… :) But I will naturally double check this before going forward with it, and then I might also get some clarification on what the two different boxes in my cabinet are. Bridge mode can be activated through a switch in that online portal though.

perfect. Then you can close the open port on your router for sure. My Torrent client (rutorrent) shows what IP and port I’m using at the bottom, these are my VPN IP and the port I opened with the VPN provider.

I’ve closed them and everything still works the same way. So I guess the ports have just been open for anyone to say hello. A good example of one of the many areas where I get confused because I don’t truly understand all this stuff very well. I learn more every day, and I’ve gotten plenty of tips in this thread, but it makes me a bit sad that self-hosting safely requires spending a lot of time learning about this stuff, and requires continued vigilance to keep things updated. This excludes a lot of people from enjoying the freedom that comes with data ownership and control. My issue is of course not with the self-hosted solutions - the developers have done excellent work to make these tools available to people including myself, who is not an IT professional. My issue is rather that the society at large has given the major tech players carte blanche to do whatever they want for such a long time, that true privacy is so distant for most people. Some good things going on in Europe to combat this (at least against corporate malpractices), but still not nearly good enough.

Thanks again for all your answers. I really appreciate you taking the time to educate me on this stuff. It’s time for me to log off the computer now, and stare at a large screen in my living room instead. The season finale of Stargate SG-1 season 6 awaits :)

Sweet, thanks for the tip. I also have learned a lot of Linux basics from Jay through LearnLinux TV. Good educator. Did not know about The Homelab Show though, so I will definitely check that out.

Hm, the Music Assistant at least does not quite accomplish what I hope and it seems to rely on services such as Spotify or YT Music to be integrated. I couldn’t quite evaluate the LinkPlay-solution, but his comment on SD card corruption with RPis made me a bit worried for the balenaSound approach. I guess there’s a lot of write operations in such a setup, that can easily corrupt the SD cards. I wonder how often they kept failing for him - maybe it’ll end up being some sort of a “subscription fee” 😅

My wife says no more toys at the moment, but if I were to implement this, I’d probably pick up one of those Up2Streams for each room and try out the LinkPlay integration.

Then you have something to put on your list for Christmas, if that is something you celebrate :)

I don’t like this. That’s super weird and I would not trust it. I’m sure it’s “fine” but I’d hard pass on that. Set up my own 100% for sure.

Yeah, good to have my suspicions confirmed. This setup is standard where I live now, and I don’t think you can get around it. First I noticed this was a coupe of years back. I’ll start finding a suitable router and set it up in bridge mode.

I don’t understand. Can I get a pic (MS Paint or real or something) or some brand names or something? I understand if you don’t want to show, I’m just not sure what you’re saying.

I have two small boxes in a cabinet - one is receiving a white cable that comes from outside my home, and outputs an optical signal that goes into the other box. This other box also gets a coax cable from outside my home, and outputs an ethernet connection that is connected to what my ISP calls a WiFi router. This has additional LAN ports as well.

Go to this site with out your VPN on, it will tell you if you’re using your raw internet to download torrents: https://iknowwhatyoudownload.com/en/peer/

I could not access this site now, however, I’ve checked this with the torrent address detection tool on ipleak.net many times before. I recently had an issue where my real IP would show for a second if I disconnected my VPN connection manually, but I solved this. My torrent client is bound to the interface created by the VPN client. At this point I am pretty sure it is fine. But I will close the ports again.

if you ping wifi.myisp.tld what is the IP address? is it private? what if you go to http://ip.add.re.ss it should be the same thing???

The IP address is outside my network. If I try to connect directly to the IP address, it fails the certifications, I get a list of domains that are connected to the cert and am allowed to “continue and accept the risk”, landing at the same site.

yes. Bridge mode means the ISP provider router is now only for translation (IE: from coaxial/DSL/Fibre to RJ45/cat cable). You plug the ISP device into the WAN port of your own device and now your device has the public IP address and that is what your trusting to protect you.

OK, I will definitely look into this in the near future then.

as long as the ISP router is plugged into the WAN port of your router and ONLY the WAN port, then you’re safe from the ISP shenanigans.

There’s a modem connected to the WAN port, and the router/hotspot is connected to the modem. But I guess that doesn’t change anything?

I have scripts that try to update everything every hour and I’m not really worried. I’d rather a update to a new version take down my services then trust myself to login every couple days and do it manually.

I will definitely need to setup this myself then. Do you run this as cron jobs?

Thinking about the torrent thing, there’s no better way to do it. I’d personally open a static port IE 12345 and point that at the torrent client on the PC. I would not randomize it and open a massive range on your firewall just in case. Then just close the client when you’re done and know that packets for 12345 will still reach your PC, they’re just dropped there.

OK, that is basically how it is configured now. It is not randomized in the sense that it changes every time, but it is listening on a port that was randomly chosen, but it is static since configuration.

Not that I support it, but if you’re downloading more then just Linux ISOs and you’re in a country with pretty strict laws around this sort of thing, you should be using a VPN that supports opening ports. then you do not need anything open on your firewall, just to connect to the VPN when you’re ready to sail the high seas.

I do use a VPN (with port forwarding supported, but I have not activated it, which I know could affect performance, but I have not noticed anything here). Is the port opening on my router unnecessary in this case?

Wow, I didn’t expect anyone to actually answer the questions, but it is very very appreciated. Thanks a lot for taking the time to do so.

Both. Access from the internet to your devices is protected from your Router. You should only ever open ports to things you want to access outside of your house.

Ok, good. So the firewall is already configured to block everything as far as I know, except for what I explicitly allow which for the time being is only my torrent client on two different machines (randomized ports).

In a homelab setting, I believe what I want to do is expose one port to a reverse proxy and redirect traffic to local services from there. But this is one of the things that I am uncertain about because I don’t entirely understand how this works. In my head: I open port 8080 where e.g. Nginx Proxy Manager listens. This is the only port anyone can gain access to anything inside my home network, and the proxy manager will say “Hey, this traffic should be redirected to port 8096 and this traffic should go to 4533”, but no direct connections to these ports can be made from outside my own network as they are not exposed. I am vulnerable only to the extent that there is an exploit in the proxy manager itself or the services. I intend to run all services in Docker containers, so they should not have access to anything else on the server, and the volumes that are mounted are ideally read-only (but that cannot always be the case). It sounds safe enough, but again, since I am not entirely certain that my understanding is correct, there might be a massive gaping hole somewhere I am unaware of.

Is it a private IP Address you use to access (IE http://192.168.0.1 or is it some other thing?) If it’s a private IP address, that’s standard and is no problem. I’ve never heard of logging into something like a public website to open ports on your router.

No, it is https://wifi.myisp.tld. It seems to require being connected to the WiFi to work. If I’m connected to a VPN or through mobile, it will give me an error. With my previous ISP, it was a simple login with username and password with a SMS 2FA. I never attempted to login from elsewhere then, so it might’ve been likewise protected. In any case, they do it like this so that less tech-savvy people can have the support perform any necessary changes on their behalf instead of trying to guide them by phone through the local web interface. So it can be accessed without being connected to the WiFi, but I guess there are larger problems if the ISPs system has been compromised to allow this somehow.

Yes, if they support it. This is what I do and that’s exactly what’s it’s for. Sure but this would give them access to your Providers firewall, which you have your own firewall plugged into so it doesn’t matter. You will still be protected by your self provided firewall, some things will stop working (you’ll be double NATed so public services might not work) and it’ll be a clue someone changed something on your Providers firewall.

Yeah, they support bridge mode. So is this essentially enabling free flow of traffic through and completely trusting the secondary router I provide myself? And if someone then disabled bridge mode in the online interface (again assuming that this would be possible), I am not exposed as long as I have my own router following the provider’s router?

So, if you open port TCP 443 and have Nginx or a website answering requests, you now need to make sure this is as secure as possible. ANYONE ANYWHERE can talk to it.

What happens if I have a port open that nothing is listening to? Is that a security concern? For example, the ports to my torrent clients when I am not using the torrent client.

A side point is, if it’s just you that needs access to it, consider a VPN (wireguard) or overlay network (tailscale) so you only need to open one port and that will give you access to everything you need in your network.

As I mentioned above, I am considering a reverse proxy, which to my understanding also limits the number of ports open to 1. How does that compare securitywise in your opionin?

As long as you are connected, I think network adb will stay active but if you leave your network f.e., you have to re-enable it in the developer options. But don’t take my word for it. Feels like google changed this behaviour every major release with android.

Oh, if that is the case, this will not work. I’ll test it out later this week to see.

On their blog they say: This project is made possible by the awesome work of various open source projects, including Shairport Sync for Airplay, Raspotify for Spotify Connect and Snapcast for multi-room audio sync. So they “just” glue existing stuff together which leaves you with roughly the same limitations as if you would do it yourself. It might allow spotify to be used with snapcast for multiroom but as I’m a yt music user I didn’t digg any deeper.

Hmm, maybe I misunderstood it. Here is a blog post that shows how it can also use Bluetooth. To me, it sounds like that makes an app agnostic solution as long as you are fine with using Bluetooth. My understanding is that you then just connect to one of your speakers with Bluetooth when you want to cast, and you can then control which speakers the audio should play from. I will research this more. I should order a 3.5mm jack extension to the Pi Zero W, which is the only part I miss to be able to set up a proof-of-concept at home.

Nice. I don’t have much experience with VMs yet. Is USB pass-through easily configured?

The Acrylic device also seems pretty nice - I’ll dig a little deeper on that one as well. The rabbit hole becomes ever deeper… :)

For homelab, start with the absolute basics - setup a firewall, and make sure you understand how it works! Map out your network topology, even if it is just your DSL box, router & your PC + printer with a raspberry pi (or 2) for your project.

Hehe, yeah, but even here I run into a large forest of terms and concepts. For example (the questions are only examples of things I stumble upon in this process, not questions addressed at you): “Setup a firewall” - on my server or on the router? Or both? And since my router is provided by my ISP which has its settings exposed through their online portal (which I hate the thought of), how does that factor in? What use is a router firewall if someone gains access to this portal and can configure at will? Can I set up the router in bridge mode and incorporate my own router, and thus have complete local control of my network? Couldn’t someone simply deactivate this in the online portal if they gained access there? And if I open ports in the firewall for a specific application, what risks am I running outside of exploits in the applications themselves? For example, I have opened a port in the router settings for torrenting Linux ISOs (for a specific local IP) - could traffic through the same port be used to compromise the network in other ways? etc. etc.

Suddenly I have fifteen questions. So when trying to research the answer to these questions, I often get slapped with five concepts I either barely have grasp of or don’t know at all in one sentence that tries to explain what is going on. It’s not that it is impossible to learn this way, but it tends to quickly become overwhelming, and I run into explanations of concepts I don’t have enough prerequisites to learn properly yet. Which is why I am trying to get a coherent introduction to all the topics in a sensible, curated way to beef up my understanding of it, so that the research process becomes easier.

For more depth, but not crazy detail, try the O’Reilly books or similar on networks & related security topics (there are a few!)

Cheers, I will have a look to see what I find :) Sounds very much what I am looking for.

One more thing - install Wireshark and get a real-time view of what your network is doing. Massively helpful tool.

I have been meaning to install it to learn more about the structure of the packages that are sent and received. Thanks for the reminder.

It would be nice to have other people being able to use it, but it is not a top priority for me. Also battery drain is not that much of an issue as it will only be used while at home. The last point is a bit more concerning though, so I will see if I can test this out and see how well it works. And yes, the connection seems to be the biggest issue here. But it seems that once configured, it only requires running ‘scrcpy’ on the recieving end. And KDE Connect can be setup to run commands remotely, for example I just set it up to open VS Codium in a specific folder from my phone. I can’t seem to add that command as a quick access tile (which would be my preferred option), but I could add it as a widget on my home screen for quick access so that conncetion is a button press away. How long the connection can stay for I don’t know, but I will see if I can’t test that out this week.

Did you check out balenaSound by the way? If so, what difficulties did you run into that made you discard it?

fcast looks nice, but if I understand it correctly it would require implementation in every specific application. I think if I were to jump onto the Grayjay-wagon, that could be nice, but I would love for my solution to be app agnostic.

But a lot of the underlying technology is unchanged for a long time, right? So to get a deeper grasp of e.g. different protocols and data flows, I imagine it could at least be a good starting point.

The Wikipedia-article on the Internet protocol suite of course provides a lot of information on this, but my issue with learning from Wikipedia is that it provides a long article on one topic with tons of links, but often no natural flow to the next topic. This could lead to reading up on things in the wrong order, making the learning process more difficult that it has to be. A text book on the other hand, if written well, is more a curated set of texts that introduces topics in a logical order, so that a topic is not introduced until the required prerequisites have already been treated.

A YouTube-channel (or perhaps rather a YouTube-playlist) can also provide such a curated set of material, although in my experience these videos can often be a little to superficial to get any proper understanding of the subject. There are of course exceptions.

Nice, the TOC seems promising for several of the things I am looking to learn more about.

Ok, thanks for clarifying that - as I feared. I elaborated on my progress into finding a solution to this below, in case you have any interest in it or any thoughts that could help me along.

Hehe, and for that I apologize! In case you are interested, here are the options I am so far considering. Still very much in the research phase, trying to figure out my specs before I buy any gear I don’t already have. I am not very experienced with this, so feel free to point out any baloney in the below text. Anything that could spare me time researching dead ends is gratefully received.

Option 1: Snapcast

Since I will be running a Raspberry Pi 4 with Home Assistant anyway (not yet set up properly), I would like to make use of the Snapcast integration in HA to run the Snapcast server and then set up a Raspberry Pi Zero W (with some 3.5mm extension) with all speakers I want to connect. Ideally these Zeros could be powered by the speaker themselves (through USB for example) to avoid two plugs, but I don’t know how realistic it is to achieve this, and I have not done much research into this yet.

The issue that I so far have is that I don’t know how I can stream audio from my Android device to the Snapcast server. From my understanding, and what I hoped to clarify with this thread, is that it requires a specific audio source that the Snapcast can recieve audio from. Here is a list. This seems to my limited experience much more doable from my laptop running Linux than from my Android device. But I don’t know…

Currently I will be investigating whether I can use audio streaming in scrcpy to stream audio to the Pi, and then route that via PulseAudio to Snapcast. I don’t know yet if this is a really cool idea or an incredibly stupid idea. I want to setup scrcpy for another purpose anyway, so why not try? :) It might introduce additional latency from my device to the speakers, but as this happens before the “distribution” from the server to the clients, I don’t think this would affect the synchronization. Also, I will never use this to speak on the phone with anyone, so that there is some latency doesn’t really matter to me. The biggest issue would be toggling this off and on - maybe via a remote command with KDE Connect or something like that. If I could set that up as a custom tile in the quick access menu in Android, that might work.

All in all it seems a bit too convoluted though, so I don’t have too much faith in this.

Option 2: balenaSound

So this is the solution I first learned about as an alternative to Sonos, but I was turned off by the need to connect my devices via the internet to the balenaCloud hosted by the developers. However, either I missed this in the first round of research, or they have released it since, but OpenBalena exists which has much of the functionality of balenaCloud (but not all) and can be self-hosted. If I could get the server to run on my Raspberry Pi 4, and then flash balenaOS onto each Pi Zero W, this could provide what I want as I understand that it allows to stream audio directly to the Zero Ws via Bluetooth (with subsequent sync to the OpenBalena server via WiFi). It would be a much simpler solution than the one above, especially in terms of toggling it on and off on my Android device.

A downside to this solution though, is that I believe I would not be able to install it on a Pi running Home Assistant OS (correct me if I am wrong), and that running HA through Docker makes installing new integrations a bit more cumbersome? Maybe that will pose no problem, as I don’t plan on using too many integrations anyway (Zigbee, Netatmo and a MQTT broker). I could perhaps also run a VM that runs Home Assistant OS?

Hehe, different use cases :) I typically prefer not to have things like microphones if I don’t have a use for it, particularly on devices connected to the internet like the Sonos one (afaik) is.