> Nothing came of the discussions with google. Demands by Google for changes to get the permission granted were vague, which makes it both arduous to figure out how to address them and very unclear if whatever I do will actually lead to success. Then more unrelated work to stay on play came up (dev. verification, target API level), which among other influences finally made me realize I don't have the motivation or time anymore to play this game.
I don't think Google was ever buy a "I don't want to use file APIs because writing the code would be hard." excuse for a security issue. I don't know what kind of exact "discussions" were possible here for "give me access to all user data, photos and everything because I don't think I want to use SAF APIs". It's like that dude in your company that will have a meltdown in PRs over his better way instead of fixing the comments and having code submitted.
Apple won't let you write into random directories past their APIs either, just because it would be too hard to use ObjC/Swift.
There's going to be loud, destructive friction when a 10-15 year old platform reduces the functionality available to its apps. Security models do need to evolve, but Android was introduced as a platform suitable for deep personal customization with few mandatory boundaries.
This was a competitive distinction against Apple's closed "safety first" platform design in iOS and led to an ecosystem of applications that took advantage of all these extra possibilities. As Google tightens its grip over the platform and pursues more aggressive limitations for security reasons (and whatever other ones), it's inevitable that many publishers and users are going to be deeply frustrated when the features that made their device theirs are no longer available.
(And incidentally, the restrictions on the Apple side have nothing to do with the application development language. I don't know where you would get that idea from or how to even make sense of it. It's just the nature of Apple's original design philosophy for iOS, where apps were deeply isolated and had no more capabilities than were necessary. Largely, Apple has been gradually adding capabilities to heavily-restricted apps over the lifetime of iOS, while Google has been moving in the opposite direction because they each started from opposite ends of the design space.)
Two examples that recently bothered me:
* I can't grant an application permission to read and store files in existing folders. This means I can't store file transfers through KDE connect in my Downloads or Documents folder.
* I can't access the Android/ folder without a PC at hand. This means I'm unable to mod Android games without my laptop. Not a big deal - but it's still frustrating.
4 years ago, there would have been zero friction for these use cases.
> I can't grant an application permission to read and store files in existing folders. This means I can't store file transfers through KDE connect in my Downloads or Documents folder.
That's not true at all though, `ACTION_OPEN_DOCUMENT_TREE` gives app permanent access to shared directory using the file chooser dialog. It needs no extra privileges or permissions outside user choice of dir.
Correct. A big part of the issue here is that Syncthing's Android app is more of a wrapper over the desktop app than an actual native Android implementation, so a lot of the code assumes the availability of certain features (like full filesystem access, and the ability to run continuously in the background[1]) that are suboptimal to have on a modern mobile OS.
Android has been pushing to restrict the usage of such features for a while. It sounds like now they finally pushed hard enough that Syncthing broke.
As a longtime Syncthing user I'm personally fine with this. I think its fair for Google to demand a certain minimum bar of polish for apps on the Play Store. I'll continue to use Syncthing on F-Droid so long as its feature set continues to make it superior to those more polished alternatives. Hopefully the absence of Syncthing on the Play Store will create an opportunity for another file syncing app to fill that void, or incentivize contributors to eventually bring Syncthing up to snuff to get back on the Play Store. (Or possibly, incentivize Google to develop tools to make it easier for low-budget apps like Syncthing to meet their quality standards.)
Most changes seem to come down to "the sandbox APIs aren't sufficiently isolated, let's lock down the sandbox further".
Most file system API changes aren't exactly recent, they're just inconvenient. Most changes were introduced in Android 11, with some going back all the way to Android 4.4. App developers tried to use workarounds, exceptions, and special permissions to work around API restrictions as long as they could and now the holes in the sandbox are finally being closed.
The Syncthing for Android app hasn't had much development in the past few years, so years of minor changes have added up to tech debt that's (too) expensive to fix.
Google's app process requires active developers and just makes it plain impossible to make an app and have it work with minimal updates. You're not allowed to "feature complete" an app and just exist. Every few months they threaten me to upgrade this, upgrade that, fill out this form, submit this info and I eventually gave up this year and they've already deleted my developer account and removed the app from search.
I feel like theyr'e doing this just to minimize storage costs or something lol. Android dev sucks for a hobbyist
It sucks for small software entrepreneurs too, as the cost of keeping a trustworthy developer on retainer for that kind of maintenance work can easily eat the modest revenue for a good niche app. And iOS is fundamentally no better.
It's why both App Stores are now dominated by corporatized growth chasers compromising their UX with endless feature treadmills and pushing for subscription IAP to fund it all.
Building a personal/family lifestyle business from the long tail on a few good niche apps, sold at a modest and respectful upfront cost, is pretty much a thing of the past now; and all the software we loved has been delisted or sold to those corporatized growth chasers.
Yeah, I'm sure it can. The app I was talking about was an app that handled appointments for my mom's business. 80% of her customers used the mobile website but a few liked the app for notifications and just liked apps but I eventually gave up as I'm not a fulltime android dev, just a backend engineer that can hunt and peck my way through an android app. It was fine for many years but the past 2 have been horrible and I eventually told my mom I give up
Put a lot of scary warnings around it then. It's for the user to decide if they want to take the risk or not. Google took something that solved real problems better than any alternative could, did so for many years, and destroyed it for no real reason other than to further tighten control they have of the supposedly "open" platform.
They did, the upstading app developers like this one just forced people to give them full access to all data in the app (or the app wouldn't run) and ended up not implementing scoped storage - something HNers outright demanded several times and exposed as a good upside of iOS.
So stick had to come out. The full filesystem access is now reserved for apps that manage full filesystem (e.g. file explorers) and that's it. Scoped storage APIs were introduced in 2013, 11 years ago and Play started enforcing them in 2020, so the experiment with scary warnings was running for 7 years and developers refused to give up on that sweet full private file access.
It's my phone. It's my data. It's my choice to install the app. It's my choice to grant the permissions to all files. Because guess what, I'm using the app to sync all my files.
I really can't agree with Google in this particular case.
I couldn't agree more. Given how much frigging hoops I had to jump through to get my Obsidian over syncthing setup to sync with my company iPhone - I nearly gave up.
I grew up when computers didn't babysit me and tried to act like the good old GDR, knowing every thing better than their citicens.
Nowadays, I feel more and more hindered by computers, not enabled. Computers used to be a production device (I could create things with them).
Phones are not a computer - phones are a just "consume like we want you to" device.
The problem is, I want my phone to be a creation device. A device that allows me to create content, text, to do lists, shopping lists, ideas and store them. And(!) sync them using the tools I decide to use. And not force me to use tools I friggin hate, because they just don't get the job done.
I gave up. My phone now is just a communication and utility device, and thus I don’t feel the urge to upgrade until it can’t do those tasks. I went back to computers (and Linux) to be able to just use them as a computer.
Same. I wish there were an alternative (a practical pocket computer), but there really isn't. So I too gave up on fighting my phone, and have also completely stopped doing mobile development. I now treat my phone essentially as an untrusted, prepackaged walled garden with limited utility. :-/
How did you get iPhone Syncthing + Obsidian working? I was under the impression that it was basically impossible to share Möbius Sync's directory with Obsidian.
Got it working myself. I set up a share inside of Mobius Sync that reaches into the Obsidian folder. (note the entire thing, just one vault). I think there was a popup saying it was unsupported but I haven't had any problems yet.
The java.io.File API isn't removed from Android, nor inaccessible. You can absolutely still use it. Google Play has chosen to not accept it on their store unless you justify it (to their non working bots). In this case, the dev chooses to just drop the entire app because maintaining it just for Fdroid feels pointless.
There's very few permissions on android that are system/privileged/preinstalled.
That’s still not your product though. You only bought a licence.
Flash your favorite open firmware, enjoy and let regular users who cannot do that avoid permission extortion. The world has needs and issues, it is not spinning around your skillset.
Have you considered that it's a plural "you" that you're choosing to pit yourself against, with different people each weighing different complaints?
Almost by definition, the people who argue strongly for free use of their hardware and software are almost never the same people who argue strongly for safety and security restrictions. You seem to be frustrated by a contradiction or inconsistency that doesn't exist.
It's true that Google can't win the hearts of both sides, but they surely know that -- you don't need to get so personally frustrated on their behalf. It's just a company with a product in a market, and the market is never going to be uniform.
It's an open-source program, it shouldn't be held to the same standards as a closed-source program that just shuttles all of your data to someone else's computer.
The risk here isn't misuse of the data, it's that some exploit is found in the code, and the additional protection of limiting its filesystem access is marginal (but nice to have).
Except SAF is slow as hell, working on multiple files means separate calls for every little thing like file size via java API. Means everything going to be VERY slow and drain battery a lot. I've seen test from 2019 where directory listing operation is 25-50 times slower in SAF
In a perfect world, what Syncthing does would be handled at the OS level and all OSes would seamlessly interoperate. In the real world the OS vendors are hostile to interoperability so the only way to get that is with a third-party tool that has OS-level access.
In reality, Syncthing is also pretty good at destroying battery life unless you go above and beyond to configure it in a way that then hampers Syncthing's ability to provide seamless file continuity between devices. I know that there will be disagreement here, but in my own opinion OS vendors are correct in being hostile to these apps under current circumstances. They could provide better ways to have third parties implement it and provide cross platform support for their platform solutions, but I've had Syncthing be the #1 cause of battery life drain on multiple devices and platforms before and it's honestly just not worth fighting with it. To get it under control relegates it to being not much better than rsync.
I think it's important to draw a distinction between OS providers and app store providers here. The fact that they're the same entity in most cases is or should be largely irrelevant.
OS vendors shouldn't make it impossible to create apps that have unlimited access to the filesystem or that suck battery in the background. There are reasons users might want to run apps that do either or both - a file indexer, for example. All the OS should do is ask the user if the app has permission to do those things.
An app store provider on the other hand might reasonably have many criteria for inclusion, such as F-Droid only allowing FOSS. This only becomes problematic when the app store in question is effectively a monopoly.
I want it to be able to browse to another app's private folder to sync the data of that app to my computer. If browsing to there it's not the job of syncthing then I need a file manager with those permissions.
For 3 years now I can't open my Downloads or Pictures folder without root, because their shitty API is unbelievely bad that it would take around 30 minutes to open a folder with that many files.
> Google took something that solved real problems better than any alternative could, did so for many years, and destroyed it for no real reason other than to further tighten control they have of the supposedly "open" platform.
Technically, the guy who inherited Syncthing Android maintenance destroyed it because he didn't want to use the file permission APIs.
Which, of course, is a reasonable decision for a maintainer to make when they're working with limited resources. But I have to say in this case I find some of the maintainer's behavior to be a bit surprising for a project as mature as Syncthing.
Or maybe this, plus Panic removing GDrive support from Transmit, plus iA dropping Android support from Writer because of it, point to a common perception that Google's API for doing things "the right way" suck to the point of unusability. If iA and Panic and Syncthing -- all who've supported Android for many years -- can't manage to make it work, then I suspect it's broken.
Google use to allow just any app to access the whole drive. That's probably too permissive. Now they've obviously swung too far the other direction, where even well intended, experienced devs are unable to work within Google's new constraints.
I guess I'm a bit at a loss about why these app developers feel they need access to things like medical records stored on the work phone of everybody at your doctor's office.
If they do need access to literally everything on the device, then it seems reasonable that they have to pass some minimum security bar. After all, several of the apps whose data they want access to are used to secure things like private medical records, classified information, etc.
At some point, the encrypted data has to be mounted as plaintext so apps can work with it. It seems reasonable to ask for some kind of permission system so that apps have to declare they need to read these files and so users can make a decision about whether to allow that access. But these developers are refusing to even ask for that permission.
We both know that first bit is a strawman, so we can move past it.
From the description of the people who wrote these apps, there are 2 basic APIs at play:
1. Get access to the entire drive.
2. Ask for permission to individual files.
I would have assumed there was a middle ground like asking for permission to a specific folder's contents, yet those same devs insist that's not the case. iA Writer users want to edit everything inside a folder. Syncthing users want to sync an entire folder. Transmit users want to select upload/download to/from an entire folder. If Google made those APIs available then we wouldn't be having this conversation.
> I guess I'm a bit at a loss about why these app developers feel they need access to things like medical records stored on the work phone of everybody at your doctor's office.
...which isn't a strawman. It's begging the question by presuming that authors actually feel such a need. I'm fairly certain the devs involved do not want or care about accessing medical records.
As to OPEN_DOCUMENT_TREE, to my naive eyes that's what it looks like to me, too. That said, I'm confident that the devs we've discussed here, particularly the ones who sell the related apps for their livelihood, are clever enough to read the docs and that they've ruled it out for some reason. I certainly don't think the Syncthing team is too incompetent to use a documented method if it magically did the right thing.
That's exactly what OPEN_DOCUMENT_TREE does. The typical workflow involves presenting a directory picker activity to the user and asking them to pick a dir (the app can then retroactively ask the OS to persist these permissions across restarts.)
However:
- You can't pick the root of a storage volume, only its subdirectories. This is presumably for your own good.
- The application is still forced to use the SAF APIs, which are slow (each call requires an IPC) and overcomplicated. For working with multiple files, directory listings, etc, naive use of SAF can be a couple orders of magnitude slower than standard File APIs. This can be sped up, but it's never going to be anywhere close to native speed.
> . It's begging the question by presuming that authors actually feel such a need. I'm fairly certain the devs involved do not want or care about accessing medical records.
iA -- one of the cases you mention -- balked at needing a standardized security assessment to access the full Google Drive.
> In order to get our users full access to their Google Drive on their devices, we now needed to pass a yearly CASA (Cloud Application Security Assessment) audit.
Googe Drive as part of Workspace is HIPAA compliant (https://support.google.com/a/answer/3407054?hl=en), meaning medical offices can and do host medical records on Drive. It's not mentioned in the article why iA needs access to all files on a HIPAA compliant service.
Apple's position is different, because Apple never let you have this kind of access.
Android/Google Play review keep restricting APIs and replacing them with less capable APIs, or keeping the APIs and reducing functionality.
It works again, but I had a USB endoscope that stopped working because Google pulled APIs and took a while to replace them. I can't use location sharing in my choice of apps anymore because something on my phone blocks either app runtime, app internet access, app location, or gps decoding and I don't use it often enough to be motivated to delve through logcat to figure it out, if they even still let me use logcat?. I'm sure it helps my battery life, but it reduces the functionality of the phone.
On that note. what is with this modern trend of trying to pretend the filesystem does not exist.
why does google(or apple) need "special interfaces" to access the filesystem in a specific way, why don't they just use the existing file api and improve the file access permission system.
I think the unix single tree filesystem was one of their great innovations and see this multi tree api fragmentation bullshit as a sort of backwards regression.
You're nearly correct, actually. In addition to security, SAF was supposed to provide a consistent interface to access files from various sources, including network sources, not just the local filesystem. Unfortunately the implementation just kind of sucks.
For the overwhelming majority of users, the file system is a confusing implementation detail that often breaks something when they're forced or tricked into directly interacting with it.
Try asking an elderly user of an Android phone where the attachments from gmail they have tried to save are stored in.
I think they'd cope a lot better with a standard folder/file hierarchy as opposed to saving into a folder/file hierarchy but never at any point telling the user where in the hierarchy they saved to!
And every app saves to a different location. I've been using computers my entire life and I generally have to resort to a file manager to find what I'm looking for.
SAF is a slow, buggy mess, and it only works in Java/Kotlin, so it's understandable that they don't want to use it. GrapheneOS manages to allow native access (via Java or machine code) to only specific user-selected directories through its Storage Scopes feature [1]. They basically did SAF but correctly and without the funding of a megacorp.
>They basically did SAF but correctly and without the funding of a megacorp.
It's also way more clunky for the user than using the file picker, and has the potential for user confusion because the app is silently denied access so it thinks the file/folder doesn't exist.
You're right, it's far from perfect, but it shows that it's not difficult to restrict native access to specific files/folders using the kernel and not weird Java IPC (which I guess should be obvious anyway). Google could've opted to provide native access to files, in addition to access via SAF, when you select a file via the picker, but they didn't. Graphene did it correctly from the low-level implementation side, not the UX side (but they can't really make the UX easier without breaking compatibility with standard Android apps AFAIK).
I think burn out on free projects is a real thing. Heck I get burn out on old projects and I’m being -paid- to maintain them. Good will and accolades only go so far and passion runs out eventually, especially if you’re only one person and there’s no one there to give you respite for a good recharge and you’re facing a hostile entity like Google.
I would have to disagree, so far the new file API has been the most buggy experience for many apps that have to use files every time that the app is running or is in background, and this is from a user experience perspective alone.
I can understand why the developers can't be bothered with a badly thought out new system.
it's how the cross platform software works and has always worked. demanding a total rewrite just to publish on a single channel is insane, especially since this used to be the ONLY way to do things.
google could always contribute to the open source app to implement the features they wish to see, but instead of using their billions for good they'd rather use it for evil.
I used to develop Android professionally (at Dropbox in the 2010s, so I have some familiarity with older Android filesystem APIs) and made a very conscious decision to switch to devx and backend work and get out of Android (as did most of my former Android colleagues). The unending hoops you had to jump through and API changes to keep your app working were too much of a pain.
As a fun anecdote, in 2014 when the "secure" Storage Access Framework was new, I found a trivial directory traversal vuln that allowed writing to any app's private directory by just passing a "../../" file name to the system [0, 1]. It was so trivial I noticed it while just browsing AOSP source to understand SAF better...
Android also used to grant world execute bits to app folders for the longest time, allowing malicious apps to create hard links to other apps' files by name, which could then be handed back to that app for a confused-deputy attack to gain access to the file contents.
All that to say - I'm glad Android has been working on security, but it was built upon such a loose foundation that tons of apps used and abused that it's going to drive developers out of the ecosystem as they have to keep adapting to a continuous stream of major breaking changes as things are locked down.
It sucks that the ongoing maintenance cost for the native mobile platforms is so high. Who wants to develop on top of a platform that is constantly changing out from under you?
It really makes me nostalgic for the vision of webOS (although not the implementation of webOS from 14 years ago).
But that's Scott's point. If the OS devs had thought through this from the beginning, app devs wouldn't have to keep dealing with breakage. iOS devs have other issues, but not these.
Apple and Google approached the mobile OS from opposite sides. Apple locked everything down and has gradually been adding access/features. Google left the barn door open, and is now trying to shut it. I know which OS/API I'd rather program against.
Heh, I never worked on iOS but based on what I heard from our iOS team at the time, I don't think iOS was any better. Though a lot of the frustrations back then were largely app review issues rather than API stability, like trying to push out a big feature release or bug fix and getting rejected because the reviewer found a new way to follow 20 links to get from the help center website to a page allowing you to sign up for a subscription outside the app store...
Web might be a better counter example - it started super locked down, but has slowly gained useful functionality like notifications, USB, serial, GPU, etc within the sandbox model. It just encourages more investment over time as new functionality is added, rather than annoying devs as useful functionality (documented or undocumented) is taken away.
iOS doesn't regularly break these mainstay APIs.. but when they do break APIs, they never backport them, unlike Google.
One example of an API where we lost power in exchange for security was UIWebView -> WKWebView.
It can end up being far more annoying than usual, even for smaller APIs, because you must maintain both versions of APIs until you get the green light to raise the minimum permitted iOS version.
"Planning to close my Google Play Developer Account. Please say hi if you are interested in obtaining the latest gplay release files from me to help in publishing this app."
It's going to be much less of a problem for everyone, with these draconian Play policies, as I imagine more and more people will be installing F-droid.
I've been using that one for a long time now. I recall that when I got started with Syncthing (some months to a year or two ago?), it seemed to have been the folk wisdom to use Syncthing-Fork, but I don't recall what the reason was.
Seems to work. Exported the config of the other one and imported it in this one. Seems all the settings are there. Sync seems to work too. Kudos to the devs.
In this case the author doesn't want to use Storage Access Framework APIs to access the file system which were mandated a few years ago as the way to access data outside the app sandbox.
They're Java-only APIs and since Syncthings core isn't written in Java, the author would have to write JNI glue to call Android when writing/reading files (and honestly this would be quite tedious, because the way SAF works is to assign Uris to each file and you need to keep querying it to get folder structure since you can't just concat paths like with files).
The author didn't want to do that and tried to persuade Google to continue letting them access all files, all photos and all documents directly and Google said no. That was the "difficulty" - turns out it's really hard to publish on Play if you refuse to follow the guidelines. Just like on AppStore.
To be fair, you're making the most used mobile operating system in the world and can't be bothered to make API bindings for more than one language? Or at least make the process easy so that someone else creates them? I am not an Android developer, but that seems also part of the problem.
fopen(), fwrite(), etc. are a part of libc and as such are officially supported [1], but they become somewhat useless with SAF. (It makes sense that you'd at least have to use some Java to open a file picker and request access, but why not provide native access as well when a file is picked?)
And they can be used the same as java.io.File, inside the application sandbox.
Other than that, it boils down to:
"
- Squeeze extra performance out of a device to achieve low latency or run computationally intensive applications, such as games or physics simulations.
- Reuse your own or other developers' C or C++ libraries. "
Those are vague examples of what the NDK could be used for, not an exhaustive list of all supported use cases. Syncthing is really just trying to re-use their existing code on Android (example use case #2). It doesn't even use low-level syscalls or anything crazy like that, it just calls fopen() on files which the user has explicitly granted to the app, which you'd expect would be available within the application sandbox.
> And most likely safety, not having C and C++ dealing directly with random bytes from untrusted files
You might as well just axe the entire NDK if you're worried about unsafe code.
Yeah, I personally think Scoped Storage on Android was a bit misdesigned and it would probably be much easier for all app developers if they (ab)used FUSE to present allowlisted directories inside app-specific mounts.
Yeah, Android has an immense churn of very low quality APIs with spotty support outside JVM languages and worse documentation.
Combine that with the fact that Android users are magnitudes less willing to fund apps (either by buying them or donating), and the result is an abysmal ecosystem that does not reward continued participation in it.
To be fair, it's really messy to do Go on Android calling back into Java because of how its runtime works. I'm not surprised they don't want to do it if it's a hobby project and it'd require making substantial changes to Syncthing's core logic.
It is - the way I always structured our architecture in this case was to write as much as possible of file handling in Java side and keep JNI surface minimal (it's also better for performance).
But that's really hard to do if you didn't begin with cross platform architecture that doesn't take into account having to modularize the filesystem layer for Android/iOS :/
Since you have such strong opinions on the matter, and experience, why don't you contribute to the SyncThing android app and implement this? Alternatively you could grab your time machine, travel back several years and let them know to anticipate this arbitrary change google would pull in the future.
I professionally contribute (and have contributed) to many projects to make them compatible with Play Store policies (that's my job after all), but I have limited time and generally the attitude of SyncThing developer kinda annoys me since it's an attitude of that developer in your PR that will spend weeks of time arguing over code implementation instead of fixing the comments in a day.
I think it's because he doesn't have a time machine and doesn't have time to donate to rewrite someone else's project that the owner expressly doesn't want rewritten.
This is what we call "Put up or shut up". It's easy to bash someone for not wanting to spend many hours of their time to work they have no interest in, just because some third party is now demanding it. The change is absolutely arbitrary, also. There used to be no way to grant apps access to specific folders. This is when the app was written. This still works. Google's own apps work that way. But now Google has also implemented additional ways to access the filesystem, and they are demanding people who don't even work for them to rewrite their projects.
It would be understandable if they demanded new apps to adhere to these new policies. But blocking older apps, that were written when there literally wasn't an alternative available, to do a full rewrite or be banned from updating? Absurd.
If you are now claiming your question was rhetorical, it doesn't mean that when answered, it is snark.
> It's easy to bash someone
No one's being bashed. Lets put it in stark relief. Here's the bashing you replied to: "But that's really hard to do if you didn't begin with cross platform architecture that doesn't take into account having to modularize the filesystem layer for Android/iOS :/"
> The change is absolutely arbitrary
No, it isn't.
We can tell you know that, because you immediately say "There used to be no way to grant apps access to specific folders."
> But now
"Now" == "3 years ago"
> demanding people who don't even work for them to rewrite their projects
They're not demanding anything, other than Google demanding Google can't keep taking updates to an app they put on their store, with full filesystem access, 3 years later, unless they patch the filesystem access. As other comments note, there's plenty of storefronts for these situations.
n.b. it's dangerous to take updates with unpatched security flaws, because bad actors buy apps/browser extensions. This is roughly the minimal action Google needs to take to avoid being liable - it's been 3 years, it looks like complicity to claim its store is safe and secure, then repeatedly take updates to products on the store with known security vulnerabilities, for years.
> But blocking older apps, that were written when there literally wasn't an alternative available, to do a full rewrite or be banned from updating
Though interpretative charity, I'm guessing you mean Google won't put updates from the vendor on the store, unless the update includes stopping asking for full filesystem access with 0 alternatives.
Android being linux has exactly nothing to do with the fact that apps can't access resources outside their sandbox if they aren't mapped in somehow. Just like with lxc or docker containers on linux, jails on *bsd, or any other sandboxes.
The only linux thing is always the kernel. Userspace isn't linux by definition on any* system. That might seem like a pedantic point to make, but I would like to point out that gnu utils aren't required to make a system "Linux", and whether you're using a drop-in replacement like uutils or go a more different route doesn't make it any less "Linux".
Android Apps are and mostly always have been restricted to the Java virtual machine (or its modern equivalent) exactly because that makes them portable between sometimes quite different base systems from different vendors. If you insist that makes it less of a Linux system I'd like to know what you think of flatpak apps on top of immutable distros. I hope you agree that, conceptually, they're quite close actually.
It certainly makes it less Linux, as most people mean Linux kernel plus GNU userspace with glibc, or possibly muslc as alternative to glibc.
Nothing of that is expected to be supported on NDK, at least officially, Google and partners have the freedom to break those expectations as much as they feel like it.
I'd just like to interject for a moment. What you're refering to as Linux, is in fact, GNU/Linux, or as I've recently taken to calling it, GNU plus Linux. Linux is not an operating system unto itself, but rather another free component of a fully functioning GNU system made useful by the GNU corelibs, shell utilities and vital system components comprising a full OS as defined by POSIX.
Many computer users run a modified version of the GNU system every day, without realizing it. Through a peculiar turn of events, the version of GNU which is widely used today is often called Linux, and many of its users are not aware that it is basically the GNU system, developed by the GNU Project.
There really is a Linux, and these people are using it, but it is just a part of the system they use. Linux is the kernel: the program in the system that allocates the machine's resources to the other programs that you run. The kernel is an essential part of an operating system, but useless by itself; it can only function in the context of a complete operating system. Linux is normally used in combination with the GNU operating system: the whole system is basically GNU with Linux added, or GNU/Linux. All the so-called Linux distributions are really distributions of GNU/Linux!
And perhaps more to the point - you USED to be able to use normal Java file apis and syscalls outside of Java, but that functionality has been gradually whittled away (in the name of legitimate security improvement) over the years, meaning "basic" IO functionality your apps relied upon could be taken away at any point and replaced by less ergonomic Java-only APIs with less functionality.
Fun fact: the official Dropbox Android app used to use inotify to watch for changes to the publicly writeable synced files in order to sync back to the cloud! Had to be replaced by java Storage Access Framework APIs later.
Another fun fact is that the Android sdk came with a JNI wrapper around inotify but it buggily stored inotify handles in a static (global, within an app vm) dictionary, meaning you'd silently lose tracking if you created more than one Java file watcher object that happened to be for the same path, so I had to rewrite that JNI wrapper back in the day to avoid that global state bug.
Wait, I seem to remember this discussion from years ago and thought it was resolved. Back then, Google wanted to drop the "access all files" permission from Android's permission system entirely but IIRC Syncthing & file manager devs then convinced them to keep it. But now Google comes back at them with a Google Play policy that prevents them from using that permission in practice?
You have to ask permission via a form from Google and your app must closely fit a white list of use cases (file manager, etc.) Obviously there's a high chance they just say "nah". Definitely sucks.
Yeah, but the legitimate use-case for that permission is VERY narrow - as in, if there's ANY way to implement your use-case using SAF, you MUST use SAF.
And SyncThing can be implemented using SAF - it's "just" a lot of work and that's usually not enough for Play Store to grant exception.
> They're Java-only APIs and since Syncthings core isn't written in Java, the author would have to write JNI glue to call Android when writing/reading files
While I don't know about this developer's specific issues, I can comment on my own issues with Google Play as an Android developer. Google Play continues to become more and more stringent with app permissions and the approval of these permissions is very vague. With my own app, from one minor release the next, one day I'll receive approval for my app's permissions and the next week I will not even though only minor changes to the app have been made. When I reach out to Google Play support, the answers are always extremely vague, canned and repetitive and I never know if an update to my app will get approval or not. It's a horrible way to develop anything.
The most annoying requirement is their Play Store delete account url. We have an API where we can delete the user’s account. But no, Google wanted a stupid url.
Is it really that hard to set up a small proxy tool that calls your fancy api when it receives those requests? As an outsider, it does seem quite reasonable to me - Google couldn't possibly support all APIs there may possibly exist for every app there is.
how are you going to authenticate the user? now you need to solve that if you didn't have a web login before.
---
Guess @dang decided to rate limit my account again so I can't post replies :-)
> Some token that every account gets generated? It's really not that much to ask honestly.
How is the user going to know this token when they visit the website on their laptop? Keep in mind that the Google requirement is that you link to this delete page from the play store, where the user is not authenticated with your app. You can't just generate an URL containing this token.
Btw, if you (or anyone) don't want to be rate limited on HN, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future.
I've done a few apps as part of my day job. And my best explanation and/or analogy is government regulations. The store requirements, apis and rules are documented up to the upteenth degree in 49 pages spread across many areas, and unless you're "in the know", you have no way to implement it to a reliable degree. And then all this ends up doing is punishing small / low-budget / low-time developers, leading to consolidation around the big players.
The big players can push their weight around to some degree, they get an element of built-in trust, and they have the sheer budget/time to implement all the ridiculous and sometimes onerous requirements. All in all, they're cementing their market position and trying to make "sticky" and invested players that will prop-up the play store for the coming decades.
The KeePass2Android app gains a bit of functionality if you use it with SFTP instead. You get the ability to, for example, merge changes in the event that there's a conflict. I recommend using SFTP to a machine that then runs SyncThing to the rest of your devices.
I would think that a user competent to use and want sync thing, is perfectly capable of depending on f droid as a source for the apk.
Can that not be enough of a distribution channel.
Well, I'll be putting the APK in a safe place, along with my Turbo Pascal floppies. ;-) Syncthing for Android has been vital to managing my sheet music collection.
The app will work until Google decides to remove APIs it uses, which hasn't been put on a roadmap yet AFAIK.
This is just removing it from the play store, which is much picker about API use than the OS. The APK will continue to exist and run fine (though not developed, try a fork).
If the Catfriend fork goes, I'll have no use for Syncthing at all, and it will be a very sad day in open source. I'll probably move to some proprietary app, since I'm definitely not interested in self hosting anything, and there's not really any FOSS equivalents other than Syncthing itself.
Android really needs to just allow direct file access to any file which is under a user selected folder.
This is very likely the reason that your fork no longer wishes to be on the Play Store:
> Planning to close my Google Play Developer Account. Please say hi if you are interested in obtaining the latest gplay release files from me to help in publishing this app.
> No it doesn't, `ACTION_OPEN_DOCUMENT_TREE` provides you with a `DocumentFile`, which isn't a `java.io.File`.
You can call `contentResolver.openFileDescriptor` on a a `content://` Uri from SAF, and get a `ParcelFileDescriptor`, which can be turned into a real unix fd by calling `detach()` on it. Passing this fd into native code just works.
That's not to say that, you know, SAF is actually sane, though. `DocumentFile` lazily IPCs to some StorageProvider process every time you retrieve a property of the file (mtime, size, whatever). There are ways to speed this up by retrieving everything via the lower level `DocumentContract` API, but it definitely feels like it was poorly thought out.
>> Android really needs to just allow direct file access to any file which is under a user selected folder.
> It does.
I don't think it does for anything under Android/data/ directory though does it? Which might sound like nitpicking except that there are some apps which only store their data under Android/data with no option to put it elsewhere, which means that there is no way for syncthing to access that data.
People that aren't familiar with Android's proprietary labyrinth of a filesystem API, you mean? If you had provided evidence that it does instead of just saying so in your first revision, you probably wouldn't have been downvoted.
I'm in the same boats, I have an app on play store but that dev verification thing will cost me around 90 euros where its not making me money even though its commercially driven still, I just can't be bothered to spend extra money for Google for their company verification requirements
(ps: I tried to get support about it and was willing to provide the company info just to keep getting on the platform, but the only answers that I got were "you must use this (non free) system to have your company verified" and then I tacitly said "screw this")
For everyone using Android and fearing not to be able to use Syncthing anymore: fear not. This is only affecting the Google Play Store version of Syncthing. You can get Syncthing-Fork here on F-Droid http://f-droid.org/packages/com.github.catfriend1.syncthinga...
The catfriend fork works great (I use it) but to be clear, this post is about the official version. The official version will cease all development, it's not just limited to the Play Store.
I've been considering making a new Android app for a while. A simple one to interact with a couple of my web services with some on-device storage, nothing complicated. More than anything, the thing that's stopping me is that I _know_ if I make it, a few Android versions later, it either won't work or won't be allowed on Play. I can't predict what random piece of the Android API/policy will change, but I know something will. And I'll have to waste my time fixing something that worked fine until Google arbitrarily broke it.
I built one app before for JellyBean. I haven't been able to install it for years and I can't compile it for a new version because of a cascade of errors and required changes that I'm unwilling to do. JellyBean hadn't even reached 10 years old before my app broke, it's pathetic that app support crumbled and rotted away that quickly. It'll happen again, so I've been turned off of Android development.
I totally understand the discontentment. It makes you feel powerless.
Pretty valid criticism. But as an android dev, it's quite normal. We got used to it.
Build your app in the recommended way, don't do weird stuff, read the guidelines. Accept drawbacks. That's more or less what you have to do, then the Play Store/App Store will be happy.
Build tools are mostly good with 1-2y old code, after that, yeah be ready to upgrade a lot.
Sharing files? Probably. Reliably syncing files without interaction, and especially without requiring a central server? i don't think so.
The truth is, the android app has existed for over 10 years now and it has never been good. It kinda mostly worked for some use cases, but always-on continuous sync on battery wasn't really one of them. And that's been true even before Google justifiably started restricting filesystem access in KitKat(!) 11 years ago.
If you are looking for an Android app that does syncing in the background well, I can highly recommend FolderSync.
SFTP or SMB depending on your computer/server I guess, or any of the supported cloud options of which there are truly plenty. I'm using SFTP for everything.
FolderSync can do "on-demand" very well if you mean that you just want to tap a sync button to trigger it.
KDE Connect is great, but not for this use-case. SyncThing was absolutely the choice for automated syncing of files between Android and PC. KDE Connect just doesn't do that (AFAIK). The two tools serve pretty different purposes.
KDE Connect wasn't really reliable and easy to use while I did. And Syncthing pretty much eliminated the need to use KDE Connect for me. The only extra-feature of KDE Connect was sharing the buffer, and I just paste to Obsidian now instead (any text editor would do, but I don't know anything better, even though I hate it for being closed-source). Maybe slightly less ergonomic than KDE Connect in this regard, but it's negligible. And I don't need to send files, if I can just share files.
In fact, I think I never even connected the phone via USB since I started using Syncthing. Copying/moving to/from shared folder is amazingly both more ergonomic and much faster (I never learned, why moving files to Android device via USB is so insanely slow).
So I really don't know an alternative. It solved pretty much all my phone/PC sync needs. It also enables backuping the important stuff (I don't use Google for that, or course). Dropbox doesn't cut it either. I really don't imagine how would I use my phone w/o Syncthing, it's pretty much essential.
It's all those little convenience features that make KDE Connect a great tool for my needs. It's only the file transfer features of it that don't live up to my needs, but for that we've got SyncThing (and probably tons of other perfectly valid options that I've never tried, too). For continuous file sync / transfer, SyncThing has been just "set it and forget it" simple for me. It just "does the thing" and keeps on doin' it reliably. :)
This is one of the few reasons I was still on Android over iOS. If Android becomes just as bad as iOS why would I keep putting up with Android?
It's exhausting dealing with operating systems that treat you like a child. If that's the choice I may as well not even have a "smart" phone. There's nothing smart about it anymore. It's become a toy with a camera.
That's a bummer! Can anyone suggest an alternative way I can get files from my macOS laptop to my Android phone? I "just" want to put my folder of music files and m3us there so I can play them (with PowerAmp).
Localsend is a great program - but no alternative to syncthing. Syntcthing is running in the background and syncing automatically. Localsend needs active user interaction. Is there a way how to use localsend similarily to Syncthing?
This is disappointing. SyncThing is one of my reasons for picking Android over iPhone. I hope the author reconsidered. Would any GoFundMe-style donation goal make the hassle worthwhile?
When I migrated from Android to iPhone, syncthing was my main pain point. There is Möbius Sync, even though not open source, at least it’s an one-time small payment, which is fair considering the dev license cost. Unfortunately it’s not as reliable as the Android official app or the fork, I guess Apple is very strict with background processing, but hey it’s better than nothing.
I'm using Syncthing to sync documents between devices and I own a mac fleet and an iPhone. The Möbius Sync[1] client (no affiliation) works great for me.
The Google Play Store is a massive pain in the ass lately. For whatever reason Google wants me to continuously update my Android apps to use whatever the latest APIs are despite such apps being mostly glorified webpages.
The Apple App Store is a massive pain in the ass lately. For whatever reason Apple wants me to continuously update my iOS apps to use whatever the latest APIs are despite such apps being mostly glorified webpages.
I'm a big fan of Syncthing, and use it on Android as well as pc. But it seems they are relying MANAGE_EXTERNAL_STORAGE permission and I can't see a good reason for Google to go on permitting that in 2024. It gives an app too much power.
Android has had scoped storage for a decade now. Time to get with the program and start using the SAF.
It does feel very odd to be actually agreeing with the Goog on something...
When bad actors keep misusing and messing up user's folders & data, it was deemed necessary. I know I know, on desktop machines this is different. But they also suffer from hundreds of thousands of malware, ransomware, etc.
> An app that's supposed to sync your files having access to your files is "too much power" now?
An app that syncs your files just needs access to those file, not the entire filesystem - which is what MANAGE_EXTERNAL_STORAGE gives it. Scoped storage and the SAF would allow syncthing to do exactly what it needs to do. The problem is that the app maintainer is unwilling to do the work to bring this app into the modern era.
> It's hard to fathom just how much damage smartphones have done to personal computing, but statements like these are a grim reminder.
Grim? Really? Personal devices containing highly personal information, coupled to a virtually friction-free global app marketplace, need on-device and supply-chain protections. As an Android dev I can find plenty to be critical of when it comes to Google's api designs and app review criteria. But in this case I agree with them.
I tried Termux and found the application files I wanted to rsync aren't visible in Termux, except maybe by rooting the phone. But I need to use banking and auth apps daily on it, so I assume that isn't an option.
You have to use `termux-setup-storage` to gain access to shared storage. If you're referring to an app's private app data folder, there's no way to access that without root via any app.
that would require you to set up a domain name and use DDNS and trust your registrar, or to just always have your computers at a static IP with no NAT blocking you whatsoever.
Thank you google for killing yet another great app. There should be special category for apps that are "Done, Complete, Perfect as is", as with decent backwards compatibility on Android there should be no issues with them. I bet apk compiled for Android 1 still runs perfectly on Android 15.
The problem, as most of the time, are bad actors. Which is the reason some permissions, SAF, disallowing targeting lower SDK versions etc were introduced on the Play Store.
Depends on the exact functionality. I'm aware of some battery events you could listen to from the manifest file in 2, that were removed at some point after that and now you have to listen for them from a running service. It broke a bunch of battery monitoring apps when it happened.
Also with services, there's additional requirements to keep them running that weren't there early on.
Android is one of the few platforms on earth where anything interesting actually could be done and finished, since they have actual stable API levels unlike Linux.... And they don't allow it
What the heck, I literally come from upvoting another submission's comment about combining LocalSend with Syncthing [1], because the idea seemed great...
Once again, Im also in the niche of avoiding pay-for dozens of small services and just getting my simple stuff synced. Bam, another blowback. Im eager to ready what alternatives HN crowd might come up with....
I am developing and supporting game engines for over a dozen years. It works pretty much on any device which can run games.
Supporting it for Android is the worst experience ever. This is by far the least reliable OS in terms of compatibility or changes they make. It gets more convoluted and crapier with every exec bonus and 'feature' google invents.
I used Syncthing for a while between various Linux distros, and I used Syncthing-Fork on my Android tablet, and it was okay when it worked, but it often borked up, and there were so many arcane settings and weird failure modes. I realized that the only reason I was using Syncthing was because it appealed to the vestigial, ultra-paranoid crypto-fascist BOFH in me, and I had grown out of those attitudes.
So today I just use Google Drive and MS OneDrive like a normal person. They work great. I love 'em. They don't fail like Syncthing. They're way more secure, and fully supported. Come join me! The water's fine!
Syncthing is peer to peer, the files are already on the device. There's no way requiring one more device to be secure (the server) is better than not requiring it.
...Does the author not understand that the Google Play Store is not the only distribution mechanism for apps? Why not just continue to distribute APKs for users to install manually?
I suspect it could be, however it sounds like the author has lost all motivation to continue work on syncthing android and has formally announced that no further development will be done. And as much as I like syncthing-android I appreciate this sort of straight forward communications more and salute the author for clearly stating intentions.
Now seeing as it is open source (hooray) The way I think that will go is as follows, we will continue using the existing apk(I get mine off f-droid) for a few months to years, in the meantime seeing as the app is so useful to so many a few forks will arise, one will end up being the best, and we will eventually start using that one.
Presumably because most people don't know about F-Droid, and there's a question of whether it's worth continuing to develop the app for a tiny subset of the original audience.
they do, but google play is the main distribution channel for android apps, and without that many people will not use it (and many will complain). from the actual announcement:
> without Play releases I do no longer see enough benefit and/or have enough motivation to keep up the ongoing maintenance
They do set how permissions on upgrades of Android work so even if it’s on f-droid, when they enforce the SAF protocols fsync will stop working if you keep your Android up to date
From the (current) final comment at https://github.com/syncthing/syncthing-android/issues/2064
> Nothing came of the discussions with google. Demands by Google for changes to get the permission granted were vague, which makes it both arduous to figure out how to address them and very unclear if whatever I do will actually lead to success. Then more unrelated work to stay on play came up (dev. verification, target API level), which among other influences finally made me realize I don't have the motivation or time anymore to play this game.
I don't think Google was ever buy a "I don't want to use file APIs because writing the code would be hard." excuse for a security issue. I don't know what kind of exact "discussions" were possible here for "give me access to all user data, photos and everything because I don't think I want to use SAF APIs". It's like that dude in your company that will have a meltdown in PRs over his better way instead of fixing the comments and having code submitted.
Apple won't let you write into random directories past their APIs either, just because it would be too hard to use ObjC/Swift.
There's going to be loud, destructive friction when a 10-15 year old platform reduces the functionality available to its apps. Security models do need to evolve, but Android was introduced as a platform suitable for deep personal customization with few mandatory boundaries.
This was a competitive distinction against Apple's closed "safety first" platform design in iOS and led to an ecosystem of applications that took advantage of all these extra possibilities. As Google tightens its grip over the platform and pursues more aggressive limitations for security reasons (and whatever other ones), it's inevitable that many publishers and users are going to be deeply frustrated when the features that made their device theirs are no longer available.
(And incidentally, the restrictions on the Apple side have nothing to do with the application development language. I don't know where you would get that idea from or how to even make sense of it. It's just the nature of Apple's original design philosophy for iOS, where apps were deeply isolated and had no more capabilities than were necessary. Largely, Apple has been gradually adding capabilities to heavily-restricted apps over the lifetime of iOS, while Google has been moving in the opposite direction because they each started from opposite ends of the design space.)
Android is fine. There are many, many complaint syncing applications in the wild that use the sanctioned APIs.
The truth is Syncthing doesn’t have the resources to keep up with Android platform changes and Google’s review process.
Two examples that recently bothered me: * I can't grant an application permission to read and store files in existing folders. This means I can't store file transfers through KDE connect in my Downloads or Documents folder. * I can't access the Android/ folder without a PC at hand. This means I'm unable to mod Android games without my laptop. Not a big deal - but it's still frustrating.
4 years ago, there would have been zero friction for these use cases.
> I can't grant an application permission to read and store files in existing folders. This means I can't store file transfers through KDE connect in my Downloads or Documents folder.
That's not true at all though, `ACTION_OPEN_DOCUMENT_TREE` gives app permanent access to shared directory using the file chooser dialog. It needs no extra privileges or permissions outside user choice of dir.
Correct. A big part of the issue here is that Syncthing's Android app is more of a wrapper over the desktop app than an actual native Android implementation, so a lot of the code assumes the availability of certain features (like full filesystem access, and the ability to run continuously in the background[1]) that are suboptimal to have on a modern mobile OS.
Android has been pushing to restrict the usage of such features for a while. It sounds like now they finally pushed hard enough that Syncthing broke.
As a longtime Syncthing user I'm personally fine with this. I think its fair for Google to demand a certain minimum bar of polish for apps on the Play Store. I'll continue to use Syncthing on F-Droid so long as its feature set continues to make it superior to those more polished alternatives. Hopefully the absence of Syncthing on the Play Store will create an opportunity for another file syncing app to fill that void, or incentivize contributors to eventually bring Syncthing up to snuff to get back on the Play Store. (Or possibly, incentivize Google to develop tools to make it easier for low-budget apps like Syncthing to meet their quality standards.)
[1]: https://github.com/syncthing/syncthing-android/issues/1048#i...
Why is Android moving so fast?
Most changes seem to come down to "the sandbox APIs aren't sufficiently isolated, let's lock down the sandbox further".
Most file system API changes aren't exactly recent, they're just inconvenient. Most changes were introduced in Android 11, with some going back all the way to Android 4.4. App developers tried to use workarounds, exceptions, and special permissions to work around API restrictions as long as they could and now the holes in the sandbox are finally being closed.
The Syncthing for Android app hasn't had much development in the past few years, so years of minor changes have added up to tech debt that's (too) expensive to fix.
This change is a decade old :)
SAF was first released as part of Android 4.4, back in 2013.
Google's app process requires active developers and just makes it plain impossible to make an app and have it work with minimal updates. You're not allowed to "feature complete" an app and just exist. Every few months they threaten me to upgrade this, upgrade that, fill out this form, submit this info and I eventually gave up this year and they've already deleted my developer account and removed the app from search.
I feel like theyr'e doing this just to minimize storage costs or something lol. Android dev sucks for a hobbyist
It sucks for small software entrepreneurs too, as the cost of keeping a trustworthy developer on retainer for that kind of maintenance work can easily eat the modest revenue for a good niche app. And iOS is fundamentally no better.
It's why both App Stores are now dominated by corporatized growth chasers compromising their UX with endless feature treadmills and pushing for subscription IAP to fund it all.
Building a personal/family lifestyle business from the long tail on a few good niche apps, sold at a modest and respectful upfront cost, is pretty much a thing of the past now; and all the software we loved has been delisted or sold to those corporatized growth chasers.
Yeah, I'm sure it can. The app I was talking about was an app that handled appointments for my mom's business. 80% of her customers used the mobile website but a few liked the app for notifications and just liked apps but I eventually gave up as I'm not a fulltime android dev, just a backend engineer that can hunt and peck my way through an android app. It was fine for many years but the past 2 have been horrible and I eventually told my mom I give up
Put a lot of scary warnings around it then. It's for the user to decide if they want to take the risk or not. Google took something that solved real problems better than any alternative could, did so for many years, and destroyed it for no real reason other than to further tighten control they have of the supposedly "open" platform.
They did, the upstading app developers like this one just forced people to give them full access to all data in the app (or the app wouldn't run) and ended up not implementing scoped storage - something HNers outright demanded several times and exposed as a good upside of iOS.
So stick had to come out. The full filesystem access is now reserved for apps that manage full filesystem (e.g. file explorers) and that's it. Scoped storage APIs were introduced in 2013, 11 years ago and Play started enforcing them in 2020, so the experiment with scary warnings was running for 7 years and developers refused to give up on that sweet full private file access.
Granted, SAF is quite a shitty API.
It's my phone. It's my data. It's my choice to install the app. It's my choice to grant the permissions to all files. Because guess what, I'm using the app to sync all my files.
I really can't agree with Google in this particular case.
I couldn't agree more. Given how much frigging hoops I had to jump through to get my Obsidian over syncthing setup to sync with my company iPhone - I nearly gave up.
I grew up when computers didn't babysit me and tried to act like the good old GDR, knowing every thing better than their citicens.
Nowadays, I feel more and more hindered by computers, not enabled. Computers used to be a production device (I could create things with them).
Phones are not a computer - phones are a just "consume like we want you to" device.
The problem is, I want my phone to be a creation device. A device that allows me to create content, text, to do lists, shopping lists, ideas and store them. And(!) sync them using the tools I decide to use. And not force me to use tools I friggin hate, because they just don't get the job done.
I gave up. My phone now is just a communication and utility device, and thus I don’t feel the urge to upgrade until it can’t do those tasks. I went back to computers (and Linux) to be able to just use them as a computer.
Same. I wish there were an alternative (a practical pocket computer), but there really isn't. So I too gave up on fighting my phone, and have also completely stopped doing mobile development. I now treat my phone essentially as an untrusted, prepackaged walled garden with limited utility. :-/
I use Sailfish OS as my daily driver since 2013.
https://en.m.wikipedia.org/wiki/Sailfish_OS
Thanks for the info. I haven't looked at this in a while. I'll take another look!
> I wish there were an alternative (a practical pocket computer), but there really isn't.
There really is: https://puri.sm/products/librem-5.
And it's my daily driver.
Appreciate the feedback. For some reason I feel like I dismissed this as an option at one point, but that was long ago. I'll take another look!
How did you get iPhone Syncthing + Obsidian working? I was under the impression that it was basically impossible to share Möbius Sync's directory with Obsidian.
There is a new app Synctrain which does this.
Got it working myself. I set up a share inside of Mobius Sync that reaches into the Obsidian folder. (note the entire thing, just one vault). I think there was a popup saying it was unsupported but I haven't had any problems yet.
The java.io.File API isn't removed from Android, nor inaccessible. You can absolutely still use it. Google Play has chosen to not accept it on their store unless you justify it (to their non working bots). In this case, the dev chooses to just drop the entire app because maintaining it just for Fdroid feels pointless.
There's very few permissions on android that are system/privileged/preinstalled.
That's why you can install through F-Droid right?
You don't have root. Google does.
I'm not saying that's a good thing, but it's not exactly a secret when you bought it.
That’s still not your product though. You only bought a licence.
Flash your favorite open firmware, enjoy and let regular users who cannot do that avoid permission extortion. The world has needs and issues, it is not spinning around your skillset.
And yet you'll blame Android when some app steals a lot of data just like it always happens on this site.
Have you considered that it's a plural "you" that you're choosing to pit yourself against, with different people each weighing different complaints?
Almost by definition, the people who argue strongly for free use of their hardware and software are almost never the same people who argue strongly for safety and security restrictions. You seem to be frustrated by a contradiction or inconsistency that doesn't exist.
It's true that Google can't win the hearts of both sides, but they surely know that -- you don't need to get so personally frustrated on their behalf. It's just a company with a product in a market, and the market is never going to be uniform.
It's an open-source program, it shouldn't be held to the same standards as a closed-source program that just shuttles all of your data to someone else's computer.
The risk here isn't misuse of the data, it's that some exploit is found in the code, and the additional protection of limiting its filesystem access is marginal (but nice to have).
So Google drive doesn’t have full filesystem access anymore?
I mean, syncthing is exactly the kind of app I would expect to have full filesystem access.
Why? I want it to have access to the folders I want it to sync, not everything on my device.
Seems like a perfect fit for SAF.
> Seems like a perfect fit for SAF.
Except SAF is slow as hell, working on multiple files means separate calls for every little thing like file size via java API. Means everything going to be VERY slow and drain battery a lot. I've seen test from 2019 where directory listing operation is 25-50 times slower in SAF
https://issuetracker.google.com/issues/130261278
In a perfect world, what Syncthing does would be handled at the OS level and all OSes would seamlessly interoperate. In the real world the OS vendors are hostile to interoperability so the only way to get that is with a third-party tool that has OS-level access.
There are Android APIs to let syncthing integrate as cloud provider itself and the author didn't use those either.
In reality, Syncthing is also pretty good at destroying battery life unless you go above and beyond to configure it in a way that then hampers Syncthing's ability to provide seamless file continuity between devices. I know that there will be disagreement here, but in my own opinion OS vendors are correct in being hostile to these apps under current circumstances. They could provide better ways to have third parties implement it and provide cross platform support for their platform solutions, but I've had Syncthing be the #1 cause of battery life drain on multiple devices and platforms before and it's honestly just not worth fighting with it. To get it under control relegates it to being not much better than rsync.
I think it's important to draw a distinction between OS providers and app store providers here. The fact that they're the same entity in most cases is or should be largely irrelevant.
OS vendors shouldn't make it impossible to create apps that have unlimited access to the filesystem or that suck battery in the background. There are reasons users might want to run apps that do either or both - a file indexer, for example. All the OS should do is ask the user if the app has permission to do those things.
An app store provider on the other hand might reasonably have many criteria for inclusion, such as F-Droid only allowing FOSS. This only becomes problematic when the app store in question is effectively a monopoly.
I use syncthing to back up my entire Android phone.
At least the parts it can access, anyway: /storage/emulated/0
I want it to be able to browse to another app's private folder to sync the data of that app to my computer. If browsing to there it's not the job of syncthing then I need a file manager with those permissions.
No. Not in my world. Because I actually want to be able to backup my phone - not only demarcated parts of it.
I want to be able to get all data from my phone - regardless of what it is and what app put it there.
If I decide to only sync specific folders. So be it. But I want to be able to sync "/"
For 3 years now I can't open my Downloads or Pictures folder without root, because their shitty API is unbelievely bad that it would take around 30 minutes to open a folder with that many files.
> Google took something that solved real problems better than any alternative could, did so for many years, and destroyed it for no real reason other than to further tighten control they have of the supposedly "open" platform.
Technically, the guy who inherited Syncthing Android maintenance destroyed it because he didn't want to use the file permission APIs.
Which, of course, is a reasonable decision for a maintainer to make when they're working with limited resources. But I have to say in this case I find some of the maintainer's behavior to be a bit surprising for a project as mature as Syncthing.
Or maybe this, plus Panic removing GDrive support from Transmit, plus iA dropping Android support from Writer because of it, point to a common perception that Google's API for doing things "the right way" suck to the point of unusability. If iA and Panic and Syncthing -- all who've supported Android for many years -- can't manage to make it work, then I suspect it's broken.
Google use to allow just any app to access the whole drive. That's probably too permissive. Now they've obviously swung too far the other direction, where even well intended, experienced devs are unable to work within Google's new constraints.
I guess I'm a bit at a loss about why these app developers feel they need access to things like medical records stored on the work phone of everybody at your doctor's office.
If they do need access to literally everything on the device, then it seems reasonable that they have to pass some minimum security bar. After all, several of the apps whose data they want access to are used to secure things like private medical records, classified information, etc.
At some point, the encrypted data has to be mounted as plaintext so apps can work with it. It seems reasonable to ask for some kind of permission system so that apps have to declare they need to read these files and so users can make a decision about whether to allow that access. But these developers are refusing to even ask for that permission.
We both know that first bit is a strawman, so we can move past it.
From the description of the people who wrote these apps, there are 2 basic APIs at play:
1. Get access to the entire drive.
2. Ask for permission to individual files.
I would have assumed there was a middle ground like asking for permission to a specific folder's contents, yet those same devs insist that's not the case. iA Writer users want to edit everything inside a folder. Syncthing users want to sync an entire folder. Transmit users want to select upload/download to/from an entire folder. If Google made those APIs available then we wouldn't be having this conversation.
> We both know that first bit is a strawman, so we can move past it.
Privacy and security are literally the reasons for these APIs. I don't see how you could possibly call that a strawman.
> I would have assumed there was a middle ground like asking for permission to a specific folder's contents,
Isn't that what OPEN_DOCUMENT_TREE does? https://developer.android.com/reference/android/content/Inte...
You're right; my bad. What you said was:
> I guess I'm a bit at a loss about why these app developers feel they need access to things like medical records stored on the work phone of everybody at your doctor's office.
...which isn't a strawman. It's begging the question by presuming that authors actually feel such a need. I'm fairly certain the devs involved do not want or care about accessing medical records.
As to OPEN_DOCUMENT_TREE, to my naive eyes that's what it looks like to me, too. That said, I'm confident that the devs we've discussed here, particularly the ones who sell the related apps for their livelihood, are clever enough to read the docs and that they've ruled it out for some reason. I certainly don't think the Syncthing team is too incompetent to use a documented method if it magically did the right thing.
That's exactly what OPEN_DOCUMENT_TREE does. The typical workflow involves presenting a directory picker activity to the user and asking them to pick a dir (the app can then retroactively ask the OS to persist these permissions across restarts.)
However: - You can't pick the root of a storage volume, only its subdirectories. This is presumably for your own good. - The application is still forced to use the SAF APIs, which are slow (each call requires an IPC) and overcomplicated. For working with multiple files, directory listings, etc, naive use of SAF can be a couple orders of magnitude slower than standard File APIs. This can be sped up, but it's never going to be anywhere close to native speed.
I can imagine devs throwing a fit about that, then. They’re the ones who’ll get bad reviews even if it’s out of their control.
> . It's begging the question by presuming that authors actually feel such a need. I'm fairly certain the devs involved do not want or care about accessing medical records.
iA -- one of the cases you mention -- balked at needing a standardized security assessment to access the full Google Drive.
> In order to get our users full access to their Google Drive on their devices, we now needed to pass a yearly CASA (Cloud Application Security Assessment) audit.
https://ia.net/topics/our-android-app-is-frozen-in-carbonite
Googe Drive as part of Workspace is HIPAA compliant (https://support.google.com/a/answer/3407054?hl=en), meaning medical offices can and do host medical records on Drive. It's not mentioned in the article why iA needs access to all files on a HIPAA compliant service.
Workspace is also (at least partially) FedRAMP compliant (https://support.google.com/a/answer/13190816?hl=en). So similar questions arise as to whether iA needs to access federal data.
> for no real reason other than to further tighten control they have of the supposedly "open" platform
I think you're overlooking the obvious motivation of "maintain a lock on a substantial profit stream".
You mean google drive subscriptions? Are such revenues even "substantial"? AFAIK google makes the overwhelming majority of its revenue from ads.
Apple's position is different, because Apple never let you have this kind of access.
Android/Google Play review keep restricting APIs and replacing them with less capable APIs, or keeping the APIs and reducing functionality.
It works again, but I had a USB endoscope that stopped working because Google pulled APIs and took a while to replace them. I can't use location sharing in my choice of apps anymore because something on my phone blocks either app runtime, app internet access, app location, or gps decoding and I don't use it often enough to be motivated to delve through logcat to figure it out, if they even still let me use logcat?. I'm sure it helps my battery life, but it reduces the functionality of the phone.
On that note. what is with this modern trend of trying to pretend the filesystem does not exist.
why does google(or apple) need "special interfaces" to access the filesystem in a specific way, why don't they just use the existing file api and improve the file access permission system.
I think the unix single tree filesystem was one of their great innovations and see this multi tree api fragmentation bullshit as a sort of backwards regression.
> what is with this modern trend of trying to pretend the filesystem does not exist.
My cynical read is that the filesystem is user freedom, and the walled gardens don't want user freedom.
Cynical take: it puts Google Drive on a level playing-field with local storage (by making the local storage experience awful).
You're nearly correct, actually. In addition to security, SAF was supposed to provide a consistent interface to access files from various sources, including network sources, not just the local filesystem. Unfortunately the implementation just kind of sucks.
That sounds right, I would have implemented google drive as a filesystem driver, but I am not google so what do I know.
For the overwhelming majority of users, the file system is a confusing implementation detail that often breaks something when they're forced or tricked into directly interacting with it.
My experience is the opposite.
Try asking an elderly user of an Android phone where the attachments from gmail they have tried to save are stored in.
I think they'd cope a lot better with a standard folder/file hierarchy as opposed to saving into a folder/file hierarchy but never at any point telling the user where in the hierarchy they saved to!
And every app saves to a different location. I've been using computers my entire life and I generally have to resort to a file manager to find what I'm looking for.
This is not my experience with Windows users. Tree hierarchies are very natural for us humans.
SAF is a slow, buggy mess, and it only works in Java/Kotlin, so it's understandable that they don't want to use it. GrapheneOS manages to allow native access (via Java or machine code) to only specific user-selected directories through its Storage Scopes feature [1]. They basically did SAF but correctly and without the funding of a megacorp.
[1] https://grapheneos.org/features#storage-scopes
>They basically did SAF but correctly and without the funding of a megacorp.
It's also way more clunky for the user than using the file picker, and has the potential for user confusion because the app is silently denied access so it thinks the file/folder doesn't exist.
You're right, it's far from perfect, but it shows that it's not difficult to restrict native access to specific files/folders using the kernel and not weird Java IPC (which I guess should be obvious anyway). Google could've opted to provide native access to files, in addition to access via SAF, when you select a file via the picker, but they didn't. Graphene did it correctly from the low-level implementation side, not the UX side (but they can't really make the UX easier without breaking compatibility with standard Android apps AFAIK).
I think burn out on free projects is a real thing. Heck I get burn out on old projects and I’m being -paid- to maintain them. Good will and accolades only go so far and passion runs out eventually, especially if you’re only one person and there’s no one there to give you respite for a good recharge and you’re facing a hostile entity like Google.
I would have to disagree, so far the new file API has been the most buggy experience for many apps that have to use files every time that the app is running or is in background, and this is from a user experience perspective alone.
I can understand why the developers can't be bothered with a badly thought out new system.
it's how the cross platform software works and has always worked. demanding a total rewrite just to publish on a single channel is insane, especially since this used to be the ONLY way to do things.
google could always contribute to the open source app to implement the features they wish to see, but instead of using their billions for good they'd rather use it for evil.
I used to develop Android professionally (at Dropbox in the 2010s, so I have some familiarity with older Android filesystem APIs) and made a very conscious decision to switch to devx and backend work and get out of Android (as did most of my former Android colleagues). The unending hoops you had to jump through and API changes to keep your app working were too much of a pain.
As a fun anecdote, in 2014 when the "secure" Storage Access Framework was new, I found a trivial directory traversal vuln that allowed writing to any app's private directory by just passing a "../../" file name to the system [0, 1]. It was so trivial I noticed it while just browsing AOSP source to understand SAF better...
Android also used to grant world execute bits to app folders for the longest time, allowing malicious apps to create hard links to other apps' files by name, which could then be handed back to that app for a confused-deputy attack to gain access to the file contents.
All that to say - I'm glad Android has been working on security, but it was built upon such a loose foundation that tons of apps used and abused that it's going to drive developers out of the ecosystem as they have to keep adapting to a continuous stream of major breaking changes as things are locked down.
[0] Bug 18512473 fixed in https://android.googlesource.com/platform/frameworks/base/+/...
[1] Proof of concept video: https://www.dropbox.com/s/8dpd8visrttqbfo/poc.mp4?dl=0
It sucks that the ongoing maintenance cost for the native mobile platforms is so high. Who wants to develop on top of a platform that is constantly changing out from under you?
It really makes me nostalgic for the vision of webOS (although not the implementation of webOS from 14 years ago).
But that's Scott's point. If the OS devs had thought through this from the beginning, app devs wouldn't have to keep dealing with breakage. iOS devs have other issues, but not these.
Apple and Google approached the mobile OS from opposite sides. Apple locked everything down and has gradually been adding access/features. Google left the barn door open, and is now trying to shut it. I know which OS/API I'd rather program against.
Heh, I never worked on iOS but based on what I heard from our iOS team at the time, I don't think iOS was any better. Though a lot of the frustrations back then were largely app review issues rather than API stability, like trying to push out a big feature release or bug fix and getting rejected because the reviewer found a new way to follow 20 links to get from the help center website to a page allowing you to sign up for a subscription outside the app store...
Web might be a better counter example - it started super locked down, but has slowly gained useful functionality like notifications, USB, serial, GPU, etc within the sandbox model. It just encourages more investment over time as new functionality is added, rather than annoying devs as useful functionality (documented or undocumented) is taken away.
iOS doesn't regularly break these mainstay APIs.. but when they do break APIs, they never backport them, unlike Google.
One example of an API where we lost power in exchange for security was UIWebView -> WKWebView.
It can end up being far more annoying than usual, even for smaller APIs, because you must maintain both versions of APIs until you get the green light to raise the minimum permitted iOS version.
There is an Android Syncthing fork [1] which is active and 1.3K stars (for whatever that's worth).
[1] https://github.com/Catfriend1/syncthing-android
Good to know, although the Readme says:
"Planning to close my Google Play Developer Account. Please say hi if you are interested in obtaining the latest gplay release files from me to help in publishing this app."
It's on f-droid rather than google play.
I think for the people interested in using Syncthing rather than Dropbox or Google's syncing option, that's not _that_ much of a problem.
I'm always happy to see F-Droid exclusive apps. Gives Google less power over what they can do.
It's going to be much less of a problem for everyone, with these draconian Play policies, as I imagine more and more people will be installing F-droid.
F-Droid is not allowed if you use Google Advanced Security on your account. :( You have to side-load.
Hmm, what's GAS?
I've been using that one for a long time now. I recall that when I got started with Syncthing (some months to a year or two ago?), it seemed to have been the folk wisdom to use Syncthing-Fork, but I don't recall what the reason was.
For me, I'm pretty sure it had something to do with better and more granular run conditions on folders.
Weirdly enough, I cannot find either on Play right now..even though I have it (the original, I believe) installed through Play.
> Head to the "releases" section or F-Droid for builds.
It's in the description on GitHub. Get F-Droid.
Seems to work. Exported the config of the other one and imported it in this one. Seems all the settings are there. Sync seems to work too. Kudos to the devs.
Yep, that's what I use. Very happy with it for years!
Looking at the underlying thread[1], the author mentions that it's very hard to publish on Google Play
> Reason is a combination of Google making Play publishing something between hard and impossible
Can someone expand on what's going on here?
[1]: https://forum.syncthing.net/t/discontinuing-syncthing-androi...
In this case the author doesn't want to use Storage Access Framework APIs to access the file system which were mandated a few years ago as the way to access data outside the app sandbox.
They're Java-only APIs and since Syncthings core isn't written in Java, the author would have to write JNI glue to call Android when writing/reading files (and honestly this would be quite tedious, because the way SAF works is to assign Uris to each file and you need to keep querying it to get folder structure since you can't just concat paths like with files).
The author didn't want to do that and tried to persuade Google to continue letting them access all files, all photos and all documents directly and Google said no. That was the "difficulty" - turns out it's really hard to publish on Play if you refuse to follow the guidelines. Just like on AppStore.
To be fair, you're making the most used mobile operating system in the world and can't be bothered to make API bindings for more than one language? Or at least make the process easy so that someone else creates them? I am not an Android developer, but that seems also part of the problem.
Android has two official languages for userspace Java and Kotlin.
NDK is only for writing native methods, reuse C and C++ libraries, and better performance for 3D and real time audio.
Anything else, is not officially supported by the Android team.
fopen(), fwrite(), etc. are a part of libc and as such are officially supported [1], but they become somewhat useless with SAF. (It makes sense that you'd at least have to use some Java to open a file picker and request access, but why not provide native access as well when a file is picked?)
[1] https://developer.android.com/ndk/guides/stable_apis
And they can be used the same as java.io.File, inside the application sandbox.
Other than that, it boils down to:
" - Squeeze extra performance out of a device to achieve low latency or run computationally intensive applications, such as games or physics simulations.
- Reuse your own or other developers' C or C++ libraries. "
From https://developer.android.com/ndk/guides
And most likely safety, not having C and C++ dealing directly with random bytes from untrusted files.
Those are vague examples of what the NDK could be used for, not an exhaustive list of all supported use cases. Syncthing is really just trying to re-use their existing code on Android (example use case #2). It doesn't even use low-level syscalls or anything crazy like that, it just calls fopen() on files which the user has explicitly granted to the app, which you'd expect would be available within the application sandbox.
> And most likely safety, not having C and C++ dealing directly with random bytes from untrusted files
You might as well just axe the entire NDK if you're worried about unsafe code.
Yeah, I personally think Scoped Storage on Android was a bit misdesigned and it would probably be much easier for all app developers if they (ab)used FUSE to present allowlisted directories inside app-specific mounts.
Yeah, Android has an immense churn of very low quality APIs with spotty support outside JVM languages and worse documentation.
Combine that with the fact that Android users are magnitudes less willing to fund apps (either by buying them or donating), and the result is an abysmal ecosystem that does not reward continued participation in it.
To be fair, it's really messy to do Go on Android calling back into Java because of how its runtime works. I'm not surprised they don't want to do it if it's a hobby project and it'd require making substantial changes to Syncthing's core logic.
It is - the way I always structured our architecture in this case was to write as much as possible of file handling in Java side and keep JNI surface minimal (it's also better for performance).
But that's really hard to do if you didn't begin with cross platform architecture that doesn't take into account having to modularize the filesystem layer for Android/iOS :/
Since you have such strong opinions on the matter, and experience, why don't you contribute to the SyncThing android app and implement this? Alternatively you could grab your time machine, travel back several years and let them know to anticipate this arbitrary change google would pull in the future.
I professionally contribute (and have contributed) to many projects to make them compatible with Play Store policies (that's my job after all), but I have limited time and generally the attitude of SyncThing developer kinda annoys me since it's an attitude of that developer in your PR that will spend weeks of time arguing over code implementation instead of fixing the comments in a day.
> (aggro question)
I think it's because he doesn't have a time machine and doesn't have time to donate to rewrite someone else's project that the owner expressly doesn't want rewritten.
n.b. it wasn't arbitrary
> (snarky response)
This is what we call "Put up or shut up". It's easy to bash someone for not wanting to spend many hours of their time to work they have no interest in, just because some third party is now demanding it. The change is absolutely arbitrary, also. There used to be no way to grant apps access to specific folders. This is when the app was written. This still works. Google's own apps work that way. But now Google has also implemented additional ways to access the filesystem, and they are demanding people who don't even work for them to rewrite their projects.
It would be understandable if they demanded new apps to adhere to these new policies. But blocking older apps, that were written when there literally wasn't an alternative available, to do a full rewrite or be banned from updating? Absurd.
> (snarky response)
If you are now claiming your question was rhetorical, it doesn't mean that when answered, it is snark.
> It's easy to bash someone
No one's being bashed. Lets put it in stark relief. Here's the bashing you replied to: "But that's really hard to do if you didn't begin with cross platform architecture that doesn't take into account having to modularize the filesystem layer for Android/iOS :/"
> The change is absolutely arbitrary
No, it isn't.
We can tell you know that, because you immediately say "There used to be no way to grant apps access to specific folders."
> But now
"Now" == "3 years ago"
> demanding people who don't even work for them to rewrite their projects
They're not demanding anything, other than Google demanding Google can't keep taking updates to an app they put on their store, with full filesystem access, 3 years later, unless they patch the filesystem access. As other comments note, there's plenty of storefronts for these situations.
n.b. it's dangerous to take updates with unpatched security flaws, because bad actors buy apps/browser extensions. This is roughly the minimal action Google needs to take to avoid being liable - it's been 3 years, it looks like complicity to claim its store is safe and secure, then repeatedly take updates to products on the store with known security vulnerabilities, for years.
> But blocking older apps, that were written when there literally wasn't an alternative available, to do a full rewrite or be banned from updating
Though interpretative charity, I'm guessing you mean Google won't put updates from the vendor on the store, unless the update includes stopping asking for full filesystem access with 0 alternatives.
> to do a full rewrite
No ones demanding this
Many folks keep thinking that just because Android uses the Linux kernel, it is supposed to behave like GNU/Linux.
Likewise they will be rather surprised to insist in targeting iOS/iPadOS/watchOS as if they are UNIX clones.
Android being linux has exactly nothing to do with the fact that apps can't access resources outside their sandbox if they aren't mapped in somehow. Just like with lxc or docker containers on linux, jails on *bsd, or any other sandboxes.
The only Linux thing on Android is the kernel, use anything else at your peril regarding portability between updates and OEM custom forks.
https://developer.android.com/ndk/guides/stable_apis
The only linux thing is always the kernel. Userspace isn't linux by definition on any* system. That might seem like a pedantic point to make, but I would like to point out that gnu utils aren't required to make a system "Linux", and whether you're using a drop-in replacement like uutils or go a more different route doesn't make it any less "Linux".
Android Apps are and mostly always have been restricted to the Java virtual machine (or its modern equivalent) exactly because that makes them portable between sometimes quite different base systems from different vendors. If you insist that makes it less of a Linux system I'd like to know what you think of flatpak apps on top of immutable distros. I hope you agree that, conceptually, they're quite close actually.
It certainly makes it less Linux, as most people mean Linux kernel plus GNU userspace with glibc, or possibly muslc as alternative to glibc.
Nothing of that is expected to be supported on NDK, at least officially, Google and partners have the freedom to break those expectations as much as they feel like it.
I'd just like to interject for a moment. What you're refering to as Linux, is in fact, GNU/Linux, or as I've recently taken to calling it, GNU plus Linux. Linux is not an operating system unto itself, but rather another free component of a fully functioning GNU system made useful by the GNU corelibs, shell utilities and vital system components comprising a full OS as defined by POSIX.
Many computer users run a modified version of the GNU system every day, without realizing it. Through a peculiar turn of events, the version of GNU which is widely used today is often called Linux, and many of its users are not aware that it is basically the GNU system, developed by the GNU Project.
There really is a Linux, and these people are using it, but it is just a part of the system they use. Linux is the kernel: the program in the system that allocates the machine's resources to the other programs that you run. The kernel is an essential part of an operating system, but useless by itself; it can only function in the context of a complete operating system. Linux is normally used in combination with the GNU operating system: the whole system is basically GNU with Linux added, or GNU/Linux. All the so-called Linux distributions are really distributions of GNU/Linux!
The point is you can't use the regular filesystem syscalls on android, it has to go through a weird java layer
And perhaps more to the point - you USED to be able to use normal Java file apis and syscalls outside of Java, but that functionality has been gradually whittled away (in the name of legitimate security improvement) over the years, meaning "basic" IO functionality your apps relied upon could be taken away at any point and replaced by less ergonomic Java-only APIs with less functionality.
Fun fact: the official Dropbox Android app used to use inotify to watch for changes to the publicly writeable synced files in order to sync back to the cloud! Had to be replaced by java Storage Access Framework APIs later.
Another fun fact is that the Android sdk came with a JNI wrapper around inotify but it buggily stored inotify handles in a static (global, within an app vm) dictionary, meaning you'd silently lose tracking if you created more than one Java file watcher object that happened to be for the same path, so I had to rewrite that JNI wrapper back in the day to avoid that global state bug.
But it does behave like GNU/Linux where it matters, other than Google's arbitrary restrictions.
Wait, I seem to remember this discussion from years ago and thought it was resolved. Back then, Google wanted to drop the "access all files" permission from Android's permission system entirely but IIRC Syncthing & file manager devs then convinced them to keep it. But now Google comes back at them with a Google Play policy that prevents them from using that permission in practice?
You have to ask permission via a form from Google and your app must closely fit a white list of use cases (file manager, etc.) Obviously there's a high chance they just say "nah". Definitely sucks.
It probably doesn't help that Syncthing is a direct competitor to Google Drive.
Yeah, but the legitimate use-case for that permission is VERY narrow - as in, if there's ANY way to implement your use-case using SAF, you MUST use SAF.
And SyncThing can be implemented using SAF - it's "just" a lot of work and that's usually not enough for Play Store to grant exception.
> They're Java-only APIs and since Syncthings core isn't written in Java, the author would have to write JNI glue to call Android when writing/reading files
Syncthing-android is already written in Java, so shouldn't there already be JNI glue code? https://github.com/syncthing/syncthing-android
The actual file monitoring, access nad upload is handled by syncthing cross-platform Go library wrapped with the Java app.
It's an app that ships with/downloads the synching (go-based) binary. (I am pretty sure this is still the case, I looked at it some years ago)
While I don't know about this developer's specific issues, I can comment on my own issues with Google Play as an Android developer. Google Play continues to become more and more stringent with app permissions and the approval of these permissions is very vague. With my own app, from one minor release the next, one day I'll receive approval for my app's permissions and the next week I will not even though only minor changes to the app have been made. When I reach out to Google Play support, the answers are always extremely vague, canned and repetitive and I never know if an update to my app will get approval or not. It's a horrible way to develop anything.
Sounds just like Apple’s stores!
The most annoying requirement is their Play Store delete account url. We have an API where we can delete the user’s account. But no, Google wanted a stupid url.
Is that so that users who no longer have/want to use the app can still delete their account?
Yes we have a delete account option in our settings screen. But Google asking for url webpage for this scenario is just unnecessary.
Apple accepted it with no questions.
How do users use the API?
Is it really that hard to set up a small proxy tool that calls your fancy api when it receives those requests? As an outsider, it does seem quite reasonable to me - Google couldn't possibly support all APIs there may possibly exist for every app there is.
We can’t tell for sure since this url will be added to the Play Store. Problem is how to authenticate users.
how are you going to authenticate the user? now you need to solve that if you didn't have a web login before.
---
Guess @dang decided to rate limit my account again so I can't post replies :-)
> Some token that every account gets generated? It's really not that much to ask honestly.
How is the user going to know this token when they visit the website on their laptop? Keep in mind that the Google requirement is that you link to this delete page from the play store, where the user is not authenticated with your app. You can't just generate an URL containing this token.
Yes, we've rate limited your account because it posts comments that obviously break the site guidelines, such as https://news.ycombinator.com/item?id=41880814.
Could you please review https://news.ycombinator.com/newsguidelines.html and use the site as intended instead? I don't want to ban you but it's not cool when people keep posting like that.
Btw, if you (or anyone) don't want to be rate limited on HN, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future.
> Guess @dang decided to rate limit my account again so I can't post replies :-)
There are automatic rate limits that apply to everyone. IME you can still click on the comment's timestamp to be able to reply.
If you post something the moderators don't like, your account can be manually rate limited to a max of 5 comments per X (24?) hours.
Source? What does this look like?
After you've written your comment and when you click reply you get the message:
> You're posting too fast. Please slow down. Thanks.
That sounds like the automatic limit. Maybe there's a manual one as well with the same message though.
The triggering of the limit is automatic, but being subjected to the limit is something the moderators can turn on per account.
If I make a new account it'll be free of the limit until dang gets upset again.
Some token that every account gets generated? It's really not that much to ask honestly.
Yeah now it's just like developing for Apple. Have been suffering from Apple's vague and canned responses for years.
I've done a few apps as part of my day job. And my best explanation and/or analogy is government regulations. The store requirements, apis and rules are documented up to the upteenth degree in 49 pages spread across many areas, and unless you're "in the know", you have no way to implement it to a reliable degree. And then all this ends up doing is punishing small / low-budget / low-time developers, leading to consolidation around the big players.
The big players can push their weight around to some degree, they get an element of built-in trust, and they have the sheer budget/time to implement all the ridiculous and sometimes onerous requirements. All in all, they're cementing their market position and trying to make "sticky" and invested players that will prop-up the play store for the coming decades.
Regulatory capture for fun and profit!
This is the best analogy I have yet read. It perfectly sums up my experience.
Eek. That's how I get my keepass database on my phone. Anyone else in this predicament?
The KeePass2Android app gains a bit of functionality if you use it with SFTP instead. You get the ability to, for example, merge changes in the event that there's a conflict. I recommend using SFTP to a machine that then runs SyncThing to the rest of your devices.
Painful! But appreciate the tip
Re merges: that's very compelling. I've encountered that issue many times using Dropbox for syncing.
syncthing-fork has been working noticeably better for me for a couple years now: https://f-droid.org/en/packages/com.github.catfriend1.syncth...
I need a serverless approach because my machines at home are off most of the time so that discounts SFTP. I also use syncthing-fork.
Resilio Sync is pretty good although proprietary
I think ultimately KeepassDX must integrate Syncthing functionality into itself
Keepass and Joplin.
Likewise
I would think that a user competent to use and want sync thing, is perfectly capable of depending on f droid as a source for the apk. Can that not be enough of a distribution channel.
I guess we will find out in the next couple of months as people either migrate to the F-Droid based "syncthing-fork" or move to a different sync tool.
Well, I'll be putting the APK in a safe place, along with my Turbo Pascal floppies. ;-) Syncthing for Android has been vital to managing my sheet music collection.
I hate mobile computing so much.
This situation is so bad that I miss the Pocket PC Win32 developer experience
Does anyone know how long I could expect the current Syncthing Android app to continue working?
Please read the article before commenting. It is clearly stated.
Is it? I see
> The last release on Github and F-Droid will happen with the December 2024 Syncthing version.
but not even a hint as to whether that APK is likely to work in say 2027.
The app will work until Google decides to remove APIs it uses, which hasn't been put on a roadmap yet AFAIK.
This is just removing it from the play store, which is much picker about API use than the OS. The APK will continue to exist and run fine (though not developed, try a fork).
Thank you for your work. I'm sorry to see you go. I really hope Syncthing continues to have a presence on Android.
I hope it can live on inside F-Droid somehow.
It can be hosted in any app store, but because it doesn't use the correct APIs - according to Google - it will not work on newer Android devices.
AFAIK, it's fine on later devices, but Google Play specifically restricts the permission which they need (`MANAGE_EXTERNAL_STORAGE`)
If the Catfriend fork goes, I'll have no use for Syncthing at all, and it will be a very sad day in open source. I'll probably move to some proprietary app, since I'm definitely not interested in self hosting anything, and there's not really any FOSS equivalents other than Syncthing itself.
Android really needs to just allow direct file access to any file which is under a user selected folder.
[flagged]
>> Android really needs to just allow direct file access to any file which is under a user selected folder.
> It does.
No it doesn't, `ACTION_OPEN_DOCUMENT_TREE` provides you with a `DocumentFile`, which isn't a `java.io.File`.
From their docs:
> It offers a simplified view of a tree of documents, but it has substantial overhead
https://developer.android.com/reference/androidx/documentfil...
----
Your fork uses `MANAGE_EXTERNAL_STORAGE` which Google Play heavily restricts:
https://github.com/Catfriend1/syncthing-android/blob/main/ap...
This is very likely the reason that your fork no longer wishes to be on the Play Store:
> Planning to close my Google Play Developer Account. Please say hi if you are interested in obtaining the latest gplay release files from me to help in publishing this app.
https://github.com/Catfriend1/syncthing-android?tab=readme-o...
EDIT: The dangerous permission for `MANAGE_EXTERNAL_STORAGE` doesn't appear in the "App Info", which may be what you're seeing.
> No it doesn't, `ACTION_OPEN_DOCUMENT_TREE` provides you with a `DocumentFile`, which isn't a `java.io.File`.
You can call `contentResolver.openFileDescriptor` on a a `content://` Uri from SAF, and get a `ParcelFileDescriptor`, which can be turned into a real unix fd by calling `detach()` on it. Passing this fd into native code just works.
That's not to say that, you know, SAF is actually sane, though. `DocumentFile` lazily IPCs to some StorageProvider process every time you retrieve a property of the file (mtime, size, whatever). There are ways to speed this up by retrieving everything via the lower level `DocumentContract` API, but it definitely feels like it was poorly thought out.
>> Android really needs to just allow direct file access to any file which is under a user selected folder.
> It does.
I don't think it does for anything under Android/data/ directory though does it? Which might sound like nitpicking except that there are some apps which only store their data under Android/data with no option to put it elsewhere, which means that there is no way for syncthing to access that data.
> Take the downvote button away from imbeciles
People that aren't familiar with Android's proprietary labyrinth of a filesystem API, you mean? If you had provided evidence that it does instead of just saying so in your first revision, you probably wouldn't have been downvoted.
I'm in the same boats, I have an app on play store but that dev verification thing will cost me around 90 euros where its not making me money even though its commercially driven still, I just can't be bothered to spend extra money for Google for their company verification requirements
(ps: I tried to get support about it and was willing to provide the company info just to keep getting on the platform, but the only answers that I got were "you must use this (non free) system to have your company verified" and then I tacitly said "screw this")
For everyone using Android and fearing not to be able to use Syncthing anymore: fear not. This is only affecting the Google Play Store version of Syncthing. You can get Syncthing-Fork here on F-Droid http://f-droid.org/packages/com.github.catfriend1.syncthinga...
The catfriend fork works great (I use it) but to be clear, this post is about the official version. The official version will cease all development, it's not just limited to the Play Store.
Exactly, thanks for the clarification!
I don't understand. Author of the linked forum thread states that he discontinues both Google Play AND F-droid verion
I've been considering making a new Android app for a while. A simple one to interact with a couple of my web services with some on-device storage, nothing complicated. More than anything, the thing that's stopping me is that I _know_ if I make it, a few Android versions later, it either won't work or won't be allowed on Play. I can't predict what random piece of the Android API/policy will change, but I know something will. And I'll have to waste my time fixing something that worked fine until Google arbitrarily broke it.
I built one app before for JellyBean. I haven't been able to install it for years and I can't compile it for a new version because of a cascade of errors and required changes that I'm unwilling to do. JellyBean hadn't even reached 10 years old before my app broke, it's pathetic that app support crumbled and rotted away that quickly. It'll happen again, so I've been turned off of Android development.
I totally understand the discontentment. It makes you feel powerless.
Pretty valid criticism. But as an android dev, it's quite normal. We got used to it.
Build your app in the recommended way, don't do weird stuff, read the guidelines. Accept drawbacks. That's more or less what you have to do, then the Play Store/App Store will be happy.
Build tools are mostly good with 1-2y old code, after that, yeah be ready to upgrade a lot.
That's just great. It's literally the only use case I had for it. There are tons of better ways to share files on PC.
Sharing files? Probably. Reliably syncing files without interaction, and especially without requiring a central server? i don't think so.
The truth is, the android app has existed for over 10 years now and it has never been good. It kinda mostly worked for some use cases, but always-on continuous sync on battery wasn't really one of them. And that's been true even before Google justifiably started restricting filesystem access in KitKat(!) 11 years ago.
If you are looking for an Android app that does syncing in the background well, I can highly recommend FolderSync.
FolderSync with which protocol?
The main reason I use Syncthing is not to use Dropbox for sensitive stuff, which I also use since it's on-demand mobile sync is way better.
SFTP or SMB depending on your computer/server I guess, or any of the supported cloud options of which there are truly plenty. I'm using SFTP for everything.
FolderSync can do "on-demand" very well if you mean that you just want to tap a sync button to trigger it.
By on-demand I mean per file or folder. So a virtual filesytem that downloads as needed.
Do give KDE connect a try. It's great
I did, before syncthing. It... isn't great.
KDE Connect is great, but not for this use-case. SyncThing was absolutely the choice for automated syncing of files between Android and PC. KDE Connect just doesn't do that (AFAIK). The two tools serve pretty different purposes.
KDE Connect wasn't really reliable and easy to use while I did. And Syncthing pretty much eliminated the need to use KDE Connect for me. The only extra-feature of KDE Connect was sharing the buffer, and I just paste to Obsidian now instead (any text editor would do, but I don't know anything better, even though I hate it for being closed-source). Maybe slightly less ergonomic than KDE Connect in this regard, but it's negligible. And I don't need to send files, if I can just share files.
In fact, I think I never even connected the phone via USB since I started using Syncthing. Copying/moving to/from shared folder is amazingly both more ergonomic and much faster (I never learned, why moving files to Android device via USB is so insanely slow).
So I really don't know an alternative. It solved pretty much all my phone/PC sync needs. It also enables backuping the important stuff (I don't use Google for that, or course). Dropbox doesn't cut it either. I really don't imagine how would I use my phone w/o Syncthing, it's pretty much essential.
I run KDE on my "TV" and the media controls and clipboard sharing are worth it. I occasionally use my phone as a trackpad too.
^^^ Exactly this for me as well. ^^^
It's all those little convenience features that make KDE Connect a great tool for my needs. It's only the file transfer features of it that don't live up to my needs, but for that we've got SyncThing (and probably tons of other perfectly valid options that I've never tried, too). For continuous file sync / transfer, SyncThing has been just "set it and forget it" simple for me. It just "does the thing" and keeps on doin' it reliably. :)
Google is in self-destruct model
It is self-destruct only if you have competition. They don't, so it's just destruct.
This is one of the few reasons I was still on Android over iOS. If Android becomes just as bad as iOS why would I keep putting up with Android?
It's exhausting dealing with operating systems that treat you like a child. If that's the choice I may as well not even have a "smart" phone. There's nothing smart about it anymore. It's become a toy with a camera.
Sadly there is no real alternative to Android at the moment. Replicant are also just 2-3 developers afaik. https://replicant.us/
This is one the reasons I root my phone and just run an ssh server on it, I can rsync anything anywhere, and do full efficient filesystem backups.
If you don't have root, it's not your phone.
That's a bummer! Can anyone suggest an alternative way I can get files from my macOS laptop to my Android phone? I "just" want to put my folder of music files and m3us there so I can play them (with PowerAmp).
Syncthing-fork from f droid
Localsend
Localsend is a great program - but no alternative to syncthing. Syntcthing is running in the background and syncing automatically. Localsend needs active user interaction. Is there a way how to use localsend similarily to Syncthing?
That's not a dealbreaker for me, as I don't run syncthing continuously on either end, just when I'm trying to sync my music.
This is disappointing. SyncThing is one of my reasons for picking Android over iPhone. I hope the author reconsidered. Would any GoFundMe-style donation goal make the hassle worthwhile?
When I migrated from Android to iPhone, syncthing was my main pain point. There is Möbius Sync, even though not open source, at least it’s an one-time small payment, which is fair considering the dev license cost. Unfortunately it’s not as reliable as the Android official app or the fork, I guess Apple is very strict with background processing, but hey it’s better than nothing.
Just discovered the synctrain iOS beta which is extremely promising.
It's on the AppStore now... you can just download it there.
I'm using Syncthing to sync documents between devices and I own a mac fleet and an iPhone. The Möbius Sync[1] client (no affiliation) works great for me.
[1] https://mobiussync.com/
It works, and well, considering the constraints in iOS, but doesn't compare well with Androids version. Regular sync thing on Mac is awesome.
One of the top comments has a link to a fork in case you haven't seen it
The Google Play Store is a massive pain in the ass lately. For whatever reason Google wants me to continuously update my Android apps to use whatever the latest APIs are despite such apps being mostly glorified webpages.
The Apple App Store is a massive pain in the ass lately. For whatever reason Apple wants me to continuously update my iOS apps to use whatever the latest APIs are despite such apps being mostly glorified webpages.
I wonder how many people using Syncthing actually obtained it from the Play Store and not F-Droid? The whole point of it is decentralization, right?
I'm a big fan of Syncthing, and use it on Android as well as pc. But it seems they are relying MANAGE_EXTERNAL_STORAGE permission and I can't see a good reason for Google to go on permitting that in 2024. It gives an app too much power.
Android has had scoped storage for a decade now. Time to get with the program and start using the SAF.
It does feel very odd to be actually agreeing with the Goog on something...
An app that's supposed to sync your files having access to your files is "too much power" now?
It's hard to fathom just how much damage smartphones have done to personal computing, but statements like these are a grim reminder.
When bad actors keep misusing and messing up user's folders & data, it was deemed necessary. I know I know, on desktop machines this is different. But they also suffer from hundreds of thousands of malware, ransomware, etc.
> An app that's supposed to sync your files having access to your files is "too much power" now?
An app that syncs your files just needs access to those file, not the entire filesystem - which is what MANAGE_EXTERNAL_STORAGE gives it. Scoped storage and the SAF would allow syncthing to do exactly what it needs to do. The problem is that the app maintainer is unwilling to do the work to bring this app into the modern era.
> It's hard to fathom just how much damage smartphones have done to personal computing, but statements like these are a grim reminder.
Grim? Really? Personal devices containing highly personal information, coupled to a virtually friction-free global app marketplace, need on-device and supply-chain protections. As an Android dev I can find plenty to be critical of when it comes to Google's api designs and app review criteria. But in this case I agree with them.
Does anyone have any other opensource recommendations?
There's a fork called, rather creatively, Syncthing-Fork. It's available on F-Droid.
This one is way more battery conscious as well. Works great, I prefer it.
I have heard good things about git-annex [0]
But don't expect the same polished user experience as with SyncThings
- [0] https://git-annex.branchable.com/Android/
rsync or rclone on termux
I tried Termux and found the application files I wanted to rsync aren't visible in Termux, except maybe by rooting the phone. But I need to use banking and auth apps daily on it, so I assume that isn't an option.
You have to use `termux-setup-storage` to gain access to shared storage. If you're referring to an app's private app data folder, there's no way to access that without root via any app.
that would require you to set up a domain name and use DDNS and trust your registrar, or to just always have your computers at a static IP with no NAT blocking you whatsoever.
Or use Tailscale to solve both issues at once
Thank you google for killing yet another great app. There should be special category for apps that are "Done, Complete, Perfect as is", as with decent backwards compatibility on Android there should be no issues with them. I bet apk compiled for Android 1 still runs perfectly on Android 15.
The problem, as most of the time, are bad actors. Which is the reason some permissions, SAF, disallowing targeting lower SDK versions etc were introduced on the Play Store.
Depends on the exact functionality. I'm aware of some battery events you could listen to from the manifest file in 2, that were removed at some point after that and now you have to listen for them from a running service. It broke a bunch of battery monitoring apps when it happened.
Also with services, there's additional requirements to keep them running that weren't there early on.
Android is one of the few platforms on earth where anything interesting actually could be done and finished, since they have actual stable API levels unlike Linux.... And they don't allow it
Is Google treating Dropbox the same way? Sounds like it could be anticompetitive behaviour.
No, because Dropbox actually follows the API guidelines.
How about DropSync (which I find much more useful than the official app)?
It also uses the scoped storage APIs, it's under no threat.
And +1 dropsync/autosync is phenomenal, it's one of the few "basically perfect" apps in my list. Set it and forget it for years and zero issues ever.
I'm afraid I don't know, never used that app.
Welp, there goes this recommendation out the window. Back to suggesting Dropbox.
Suggest F-droid instead.
FWIW I use syncthing-fork from f-store and im quite happy with it. Development seems kinda slow lately but it's chugging along.
Can you please elaborate and link?
Shit, this was the app I used to sync my obsidian notes through multiple devices.
I just recently started using it. Going to have to switch to fork version.
Keepass and logseq for me
Without Android what's even the point? If all I have is laptop and desktop, I probably just run scheduled rsyncs or a systemd timer something.
You can run that on android? Sorry I’ve been out of the android world for a while :)
No, the OP said they'd do that if they weren't considering Android.
What the heck, I literally come from upvoting another submission's comment about combining LocalSend with Syncthing [1], because the idea seemed great...
That's gone very fast from "oh yeah!" to "oh no!"
[1]: https://news.ycombinator.com/item?id=41891983
There is a fork mentioned above.
Joining the IA Writer club on Android: https://news.ycombinator.com/item?id=41658023
Once again, Im also in the niche of avoiding pay-for dozens of small services and just getting my simple stuff synced. Bam, another blowback. Im eager to ready what alternatives HN crowd might come up with....
Syncthing is essential for syncing some very important data for me. I’d be so disappointed if it was discontinued for the platform I use.
Shit, this is the app I use to sync my obsidian notes through multiple devices. R.I.P.
I am developing and supporting game engines for over a dozen years. It works pretty much on any device which can run games.
Supporting it for Android is the worst experience ever. This is by far the least reliable OS in terms of compatibility or changes they make. It gets more convoluted and crapier with every exec bonus and 'feature' google invents.
Url changed from https://old.reddit.com/r/Syncthing/comments/1g7zpvm/syncthin..., which points to this.
Submitters: "Please submit the original source. If a post reports on something found on another site, submit the latter." - https://news.ycombinator.com/newsguidelines.html
[dead]
I used Syncthing for a while between various Linux distros, and I used Syncthing-Fork on my Android tablet, and it was okay when it worked, but it often borked up, and there were so many arcane settings and weird failure modes. I realized that the only reason I was using Syncthing was because it appealed to the vestigial, ultra-paranoid crypto-fascist BOFH in me, and I had grown out of those attitudes.
So today I just use Google Drive and MS OneDrive like a normal person. They work great. I love 'em. They don't fail like Syncthing. They're way more secure, and fully supported. Come join me! The water's fine!
How is Google Drive "way more secure" than a peer-to-peer encrypted solution?
Most of us do not have IDS/IPS/DLP tooling in our home networks, nor do we have a 24/7 on-call SOC team monitoring their SIEMs dashboards.
Google and Microsoft provision this stuff, even for consumers, with secure authentication and good protections.
Syncthing is peer to peer, the files are already on the device. There's no way requiring one more device to be secure (the server) is better than not requiring it.
It uses STUN/TURN so your first point is irrelevant.
I don't understand the second. Are you saying Syncthing is less secure?
...Does the author not understand that the Google Play Store is not the only distribution mechanism for apps? Why not just continue to distribute APKs for users to install manually?
The app is on F-Droid, which is mentioned in the article: https://f-droid.org/en/packages/com.nutomic.syncthingandroid...
I’m not sure why it can’t continue to be. Does anyone know why?
I suspect it could be, however it sounds like the author has lost all motivation to continue work on syncthing android and has formally announced that no further development will be done. And as much as I like syncthing-android I appreciate this sort of straight forward communications more and salute the author for clearly stating intentions.
Now seeing as it is open source (hooray) The way I think that will go is as follows, we will continue using the existing apk(I get mine off f-droid) for a few months to years, in the meantime seeing as the app is so useful to so many a few forks will arise, one will end up being the best, and we will eventually start using that one.
It's stated in the post. They do not think F-Droid provides enough distribution because it's not as popular as Google Play.
Presumably because most people don't know about F-Droid, and there's a question of whether it's worth continuing to develop the app for a tiny subset of the original audience.
they do, but google play is the main distribution channel for android apps, and without that many people will not use it (and many will complain). from the actual announcement:
> without Play releases I do no longer see enough benefit and/or have enough motivation to keep up the ongoing maintenance
https://forum.syncthing.net/t/discontinuing-syncthing-androi...
They do set how permissions on upgrades of Android work so even if it’s on f-droid, when they enforce the SAF protocols fsync will stop working if you keep your Android up to date