Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
The likely end of DownThemAll (downthemall.net)
257 points by deltaprotocol on Aug 21, 2015 | hide | past | favorite | 135 comments


There is no piece of software in the world harder to secure than a browser. There's almost no other piece of software where compromises have higher stakes. Further, the verdict is probably in on whether browsers should use multi-process sandboxes, and how careful they need to be about privilege-escalated Javascript, which is an enormous loophole for runtime security measures like ASLR and DEP.

Firefox's multi-process model is apparently called Electrolysis. Electrolysis apparently breaks XUL extensions.

If that's the short term cost of getting Firefox to the same level of security that Chrome is at, it seems more than worth it.

If this were an encrypted messaging application like TextSecure making an extension-breaking announcement for security, we'd have no trouble understanding the stakes. What some people seem to have a hard time accepting is that their browser is their most important encrypted messaging application.


>Electrolysis apparently breaks XUL extensions.

This is wrong. It absolutely does not break XUL extensions per-se. Add-ons will require some (moderate for most add-ons) changes when accessing out-of-process web content. A lot of add-ons may not even require changes at all, because they either do not access web content directly in the first place, or the Cross-Process-Wrappers and shims mozilla already implemented will be enough (changes might be still wise to get away from a blocking wrapper-API to the async frame script API, but that matters only for perceived performance of the browser, not for security)

DownThemAll! for example does already support e10s in the Nightly builds since quite some time, NoScript supported e10s in the past, Greasemonkey spend 9 months to add e10s support (Anthony tells in a comment to the original blog post). GM is admittedly an add-on that required a lot of changes to make it work, but it does work now apparently.

https://developer.mozilla.org/en-US/Add-ons/Working_with_mul...


I'm prepared to be wrong, as I am not an expert on XUL, Electrolysis, or the Firefox runtime.

I can however interpret what Mozilla says is the reasoning behind breaking XUL extensions, which is:

XUL extensions, among other things, get in the way of getting Electrolysis working.

That's linked from this blog post.

If things have to break in the short term for Mozilla to modernize Firefox's runtime hardening, well, that breakage seems worth it. Just based on the knowledge of what security flaws in Firefox get used for in the real world.

--- (few min later)

As an aside: if this is the definition of "XUL extensions can be made to work with e10s":

The lack of an synchronous API in the chrome side is intentional: because the chrome process runs the Firefox UI, any responsiveness problems affect the whole browser. By making the chrome process block on the content process, CPOWs break this principle and allow an unresponsive content process to make the whole browser unresponsive.

... I can kind of see where they're coming from, even w/o considering security.


If this breaks stuff that users are relying on, they'll quite happily switch to Pale Moon or turn off auto-updates to get it back, and if that means they're not protected against security issues, well, I'm sure you know just how much your average user knows about security. Chances are they won't bother to switch back once the breakage is fixed either because why would they?


Electrolysis is mostly already implemented with the XUL version of Firefox and many add-ons have been ported over to be compatible with it.

The main new change being announced recently (electrolysis and extension signing have been known about for a while now) is the deprecation of XUL in the browser in general and for add-ons in particular and the adoption of a new extension API's that are more limited in functionality (WebExtensions and whatever else they decide to do to allow more functionality than is available in Chrome). There are security implications to these changes but multi-process Firefox is not the main driver. Mozilla is giving up on XUL in favor of HTML and using this as a pretext to limit the functionality of add-ons (which many Firefox developers have wanted to do for a long time for security and stability reasons).


Extensions can, today, be asynchronous without sacrificing the more powerful model.


I'll copy and paste my comment from earlier:

> Reaching into content windows is forbidden [modulo CPOWs] in multiprocess Firefox. That alone is going to break tons of addons. This necessitates a redesign. Since a major redesign is necessary anyway, it makes sense to future-proof the architecture so that addons will work in perpetuity. Ultimately, this ends up being friendlier to addon developers, since addons will break once instead of again and again as the architecture evolves.


So repeating it makes it more true, eh?

>Reaching into content windows is forbidden [modulo CPOWs] in multiprocess Firefox.

It is not forbidden, it is just different, using frame scripts or CPOWs.

>That alone is going to break tons of addons.

It will break a ton of add-ons, it will not break a ton of other add-ons. And also the "breakage" varies and would be a in the range of learning a bit about frame scripts and then fixing up a couple of tens to hundreds lines of code, as opposed to porting your entire add-on to an entirely new WebExtensions-API, which essentially would require a rewrite, if that is even possible at all without losing too much functionality.

>This necessitates a redesign.

It does not necessitate a redesign of the Firefox extension API if that's what you mean here.

>Since a major redesign is necessary anyway, it makes sense to future-proof the architecture so that addons will work in perpetuity.

Just like Add-on SDK add-ons will work in perpetuity... Oh wait, they never did. The WebExtensions API will introduce breaking changes sooner or later if you want to keep it sane and secure, as any API does, because no API is designed perfectly with perfect future vision.

It will probably break far less often than the "open API", and that is a good thing. I'm not opposed at all to a nice, new, shiny, stable WebExtensions API for those users that are not affected by it's limitations. Actually, I hoped that Jetpack/the Add-on SDK would become such an API back when...

>Ultimately, this ends up being friendlier to addon developers, since addons will break once instead of again and again as the architecture evolves.

It is not friendly to intensionally break thousands of existing code bases to the point where you will need to almost completely redesign and rewrite your code, wiping out a ton of add-ons in the process because the new API is either too limited to port the code or because the author simply does not have the resources available to rewrite a ton of code.


But the Add-on SDK is continuing to be supported.

And the post makes it clear that this is a long-term decision, not motivated by any one change but by the sum of all of them. The removal of the traditional status bar in Firefox 4 broke a bunch of addons. Australis had add-on compatibility implications. Electrolysis has massive add-on compatibility hazards. In the future, HTML-based UI like browser.html would have huge add-on compatibility issues, as would hardened engine components that might not support XUL/XPCOM (because it's essentially impossible to support XPCOM without being Gecko).

Nobody likes breaking addons, believe me. But I think after 10 years it's clear that the current situation needs to change, or we'll be having this conversation 2 years from now all over again, and then 2 years after that, and so on as long as Firefox is around. With one last round of deprecation we can get to a sustainable future with great addons and minimal breakage.


It's not entirely clear to me from the post whether the reason for changing the add-on API is to get rid of XUL/XPCOM (which was inevitable) or to limit what extensions can do for security reasons and/or to make AMO reviews easier.

If it's really just about XUL/XPCOM, there are promises you (by which I mean Mozilla) could do to make developers happier:

1) That you'll figure out what will replace XUL/XPCOM in Firefox before you force add-on developers to make similar changes. Perhaps there's been some discussion of this at Mozilla since https://mail.mozilla.org/pipermail/firefox-dev/2015-July/003... that I'm not aware of.

2) That functionality provided by non-XPCOM-based APIs currently available to extensions will remain available, e.g. OS.File and js-ctypes.

3) That there will be some mechanism for overlays for future browser UIs, beyond simple browser buttons.

If it's about limiting what extensions can do, and not about the needs of new technology, I think a serious discussion needs to happen. There's always going to be some compromise between security and liberty, and developers would probably have understood if told that they have to write things in certain ways so that the browser itself can be more secure. But telling developers that they can't build what they want to build because Mozilla doesn't trust them to write secure code is more questionable. A bad programmer can write insecure code with any extension API. (Chrome's extension API didn't protect against http://blog.kotowicz.net/2013/07/jealous-of-prism-use-amazon...) I'm hoping for an add-on API that makes it easy to do common things in a secure way, but also provides an escape hatch for people who know what they're doing.

Finally, I think developers would be happier if you finished the APIs and then gave them two years. Now we know we have 12-18 months and that Firefox will run Chrome extensions, but we have no idea what else, if anything, we'll be able to do when Mozilla flips the switch and breaks our old add-ons. We have no idea whether we should start building on top of WebExtensions, switch to an extension + companion app, or just abandon Firefox entirely.

I would much rather this conversation in 2 years when the WebExtensions API is complete, or at least complete enough that we know what to expect from it, than play a game of duck hunt while you figure out the APIs and we figure out how to tailor our extensions to them, or if we will be able to at all.


> so that addons will work in perpetuity

How are we supposed to believe that an organization that systematically disregarded backwards compatibility in the past is suddenly going to commit to it?


> If that's the short term cost of getting Firefox to the same level of security that Chrome is at, it seems more than worth it.

Without the extensions I perhaps see no point to FF. Is secure and dead worth it?

Plus is there evidence of issues around this in the wild? Is it worth the risk of being a possible FF killer?


I'm in the same boat as you.

I use Firefox purely for a large number of quite complex extensions (which have no Chrome equivalent). If these no longer work or ever extension has a similar version in Chrome I guess I'll switch to Chromium.

Then again. Perhaps there will be some kind of long term fork of Firefox before the switch?


If my FF extensions break I'll probably switch to Edge, it's faster and will be using the same extension API. The only reason I use FF is the extensions that can't be found anywhere else.


> Plus is there evidence of issues around this in the wild?

Is there evidence of browser RCEs in the wild? Yes.


> Is there evidence of browser RCEs in the wild? Yes.

Is there evidence of browser RCEs caused by extensions?


Is there evidence of browser RCEs that work by in part by privilege-escalating browser-chrome Javascript? Yes.


Firefox hsa NoScript (disable Javascript) - while Google is motivated (by ad $revenues) to ensure Javascript executes. Yet in other threads you concretely have stated Chrome > FF.


I've just switched to Edge to really give it a chance, infuriating as it is at times.

Up to a few weeks ago I exclusively used Firefox. Installed plugins? Let me think.

AdBlock Plus, but not really, has been deactivated for months now.

Lazarus. Active, but never used in many months.

Nothing more, I think. And I'm a power user, not some grandmother who uses webmail once a month.

Firefox has lots of reasons to be used, even without plugins.


Maybe one valid point is having an internet browser that is not manufactured by an advertising company?


Albeit one that exists solely at the sufferance of the advertising industry.


I think it's more that nowadays the browser is the most important, if not the only way users interact with a computer, so any change in look/functionality is going to have an enormous impact on their digital lives.

Due to the ongoing nature of threats and exploits users also cannot opt for not upgrading, like they could for, say, a text editor, a painting program, etc. they are more or less implicitly forced to keep up with the releases, which means they are at the mercy of the browser developer's decisions a lot more than other software users.

It is understandable from a development perspective that there is a tying of functionality and security upgrades, on the other hand it is not the most user friendly paradigm around for those who prefer their customized environments not to have unintended changes.


idk, better extensions / plugins is the main reason I use firefox, easily.


I think some of this might be Firefox OS mentality creeping into the browser.

I have used Firefox since right after it stopped being called Firebird (0.8 - 0.9 days). I loved it because it seemed to be built around an aesthetic of tinkering. Coming from Konqueror, which is like the tape deck in your mom's minivan, Firefox was like a fancy hi-fi sound system. You get decent sound with the defaults, but if you really know what you're doing you can produce jaw-dropping results.

In grad school, I did some research work examining and improving a Firefox extension. I had toyed with extension writing before, but the power that extensions had over the DOM on one end and the whole browser experience on the other end was amazing.

Now, I think Mozilla developers have gotten the Firefox OS mentality and are treating the browser core like a kernel. Sure, you can do anything you want in user mode, but the kernel is inviolable.

I understand that Firefox is a Big Boy browser now. People are using it the world over; using it in corporate settings; trusting it to keep their personal information safe. That has got to put a lot of pressure on the Mozilla devs to make sure the browser is locked down as tight as can be.

I can install the Developer Edition. I can still tinker. I can customize my own experience to my heart's content. But as an extension author myself, I sympathize mightily with the developers of DownThemAll. I want to make others' browsing experiences better too, and lately I feel like Mozilla have been working against me rather than with me on this.

Firefox is actually a really good browser. They've made some pretty questionable decisions of late, but I do think their tech is as good as any browser tech out there today. But I liked Firefox not because I like their browser but because I could make it my browser, and Mozilla keep making it harder to do that. That makes me sad.


This seems like a classic case if thinking people use your software for the core features you develop. I hate to break it to you but people don't use windows for the control panel. Firefox and other browsers have become development platforms and many of their "users" don't use their platform for its 'control panel,' they use it for some useful tool built on top of it.

To use a linux kernel term, this breaks userspace (might be a sign that browsers have some serious OS envy). This is particularly bad because if many of your users use your platform for a tool that only works on an old, unsupported version (think XP), you actually make the security situation WORSE since those users don't care about security, they care about the tools they need to get their jobs done. They are still going to use those tools and you have just left them hanging out to dry from a security perspective. Talk about passing the buck.


> To use a linux kernel term, this breaks userspace (might be a sign that browsers have some serious OS envy).

No, the equivalent of breaking userspace would be breaking the Web platform. This is more like breaking the in-kernel API for .ko modules—and the official Linux policy on this is [1]. Firefox has always been much less inclined to break its internal XPCOM APIs than the Linux kernel has been to break its internal in-kernel APIs.

[1]: https://github.com/torvalds/linux/blob/master/Documentation/...


>No, the equivalent of breaking userspace would be breaking the Web platform.

Suppose someone secretly believes that the true users of my web browser are the owners of the web sites I visit. (Perhaps they own a web site or they make their living working for web-site owners, which makes it natural for them to care about the freedoms and the interests of site owners more than they care about my freedoms and interests.)

That secret belief would tend to make them overestimate the usefulness of the Javascript that the sites I visit cause to execute on my browser and to underestimate the usefulness and importance of any extensions or add-ons I choose to install on my browser.

I think the true situation is that there are two important userspaces -- one in which front-end web-app code runs and another in which extensions and add-ons run. (There is some overlap between these two userspaces.)

Also, one of the arguments being given for the need to restrict what add-ons can do is security even though most exploits of the browser are caused not by the user's installing a bad add-on, but rather by the user's visiting a bad web page. Yet I see no one on this page mentioning the latter kind of exploits (which an add-on can sometimes protect against, for example, by blocking ads).


The comparison only goes so far since extensions can't get "mainlined" into Firefox (well, except for that Pocket thing apparently).

Mozilla can just replace their API without worrying about updating every extensions their changes broke. That's someone else's problem.


The way you keep comparing firefox extensions to kernel modules seems really disingenuous, the paralels are weak at best.


I don't know if it's very smart to limit one of the most distinctive features of Firefox, the powerful add-ons available. Erodes the differences between Firefox and other browsers. If Firefox is going to be just another Chrome, people will just use Chrome.


The truth is Firefox already lost most of its market share to Chrome, so "powerful add-on" doesn't seem to be a strong enough selling point to retain users.

It is, however, a big liability. In terms of security but also stability and performances. The way extensions work in Firefox is a bit crazy, it's like opening your belly and say "here, change whatever you want". It's the equivalent of a desktop OS where any installed application runs in kernel land where they can slow down everything, crash everything, make the browser unusable...

Not only a bad written extensions can have disastrous consequences, it's impossible for the Firefox team to keep backward compatibility because they have no idea what XUL file the extensions developers are overlaying or what core JS object they've monkey-patched.


> The way extensions work in Firefox is a bit crazy, it's like opening your belly and say "here, change whatever you want".

...and my god it has been awesome. Certainly more awesome than any instability it may cause.

> It's the equivalent of a desktop OS where any installed application runs in kernel

Most major operating systems give extension capabilities to enhance them in deep ways. My Windows install has ClassicShell as a start menu replacement and DisplayFusion for multi-monitor taskbars. I have explorer plugins for Zip files, Git, and Subversion. Any one of those tools can crash and take down the whole desktop process.

What you're describing isn't a modern power user OS -- you're describing iOS. A safe secure place where nothing interesting happens; only exactly what Apple has already imagined you could want.

> Not only a bad written extensions can have disastrous consequences

My browser crashes? Hardly disasterous! And then I can just uninstall any unstable extensions.


> My browser crashes? Hardly disasterous!

If that crash leads to RCE (which, empirically, many of them do) then that isn't the characterization I would use.


That's a risk, that as an informed and adult user, I should be able to accept on my own. If I wanted a much less useful but marginally more secure browser, I'd just use Chrome.


That presupposes a bit of a false dichotomy, doesn't it? If the add-on APIs you need aren't there in the new framework, then the solution is to add them. I don't believe security and power should be in conflict--I want both!


You can't have power without the choice to use it wrongly.

You cannot add them in the new system without having the same security hole as the old system. e.g access to filesystems

You cannot have the power to access filesystems while also remaining secure. It's an all-or-nothing deal.


> You cannot have the power to access filesystems while also remaining secure. It's an all-or-nothing deal.

No, it's not. There are many secure applications that can access the filesystem.


But they have restrictions that limit their functionality, so they are trading power for security...

Imagine this. I need read/write access of the host file or an equivalent functionality of mapping any arbitrary domain name to any arbitrary IP address.

There is absolutely no way to keep that secure while allowing it because the functionality itself can be used for the wrong reasons. You either allow said functionality or you don't. There is absolutely no gray area.


> What you're describing isn't a modern power user OS -- you're describing iOS.

Hardly. Windows, Linux and OS X do not let you monkeypatch the entire system.


Actually, they pretty much do. Take for example Dropbox. Windows officially 3rd party applications adding overlays to icons which dropbox uses to provide sync status for items in the dropbox folder. OS X has no such feature or API for icon overlays. Through lots of trial and error, swizzling, mach_inject, and mach_override they managed to provide this feature in OS X as well.


> Through lots of trial and error, swizzling, mach_inject, and mach_override they managed to provide this feature in OS X as well.

And it's not unlikely that Apple will break them at any time. Apple won't even go through the trouble of making an announcement when they do so.


In fact, I believe they are doing exactly this in El Capitan. Under the umbrella of "Rootless", you will no longer be able to arbitrarily modify [1] some special blessed OS processes, like Finder.app.

Some discussion of the politics of this issue: http://atp.fm/episodes/128

[1] haven't read enough to know the exact mechanism, but presumably restricting some forms of IPC like Mach messages?


I'm sure they understood the risk when they did it.


Well I'd say the worst that can happen is an extension that compromises security of the browser to the point that malicious people can take control of your online accounts such as email, bank accounts, source code on github, etc.

That can happen intentionally but more often unintentionally. For example as you need to communicate between JS living in chrome (not the browser) and the webpage, it's easy to open a hole by mistake. Then someone could build a website to inject code to your browser and pretty much take control of your machine.

Yes Mozilla reviews the add-ons, but they are so many of them that they can't take the time to do a bullet-proof review. They only check for a few basic rules.


I've got a few plugins that provide very useful tasks but aren't very trust-worthy. You might think that's insane but everything has a risk. I do disable those add-ins when I don't need them.

Security is not an absolute -- it's a trade off.


I would wager that your attention to security as it relates to addons puts you in the minorty when compared to the Firefox user base.

It is great that you are security aware but what about the masses?


Well, from a business point of view it makes sense for them to assimilate (see Hotelling's law[0])

As a user I am unhappy about their decision. For me, the whole point of the existence of Firefox is the freedom to customize the browser in whatever way I want to. They shouldn't change that. If they really think that all their users are stupid then perhaps they could display a special warning when a "dangerous" piece of software (i.e one that uses XUL) is about to be installed.

[0] https://en.wikipedia.org/wiki/Hotelling%27s_law


I agree, if they are all going to look alike, work alike and the base is now Chrome, then might as well use Chrome. But frankly if I wanted to use Chrome then I'd be using Chrome now.

I've stopped the version madness here, I have some plugins that I love that preclude me from going forward. I'd rather see the team go "Hey we have 2.3 million person years of backlogged bugs, we are going to fix them before we make a fundamental change to the browser.

Security is an issue, but with my work machine I'm not hitting sites that would put me at risk.


> Security is an issue, but with my work machine I'm not hitting sites that would put me at risk.

How can you possibly know this for certain?


> Security is an issue, but with my work machine I'm not hitting sites that would put me at risk.

What add-ons do you use on this extremely restricted set of internal-only websites?


Visiting only trusted sites is a niche use case for web browsers.


I'll be pretty upset if the addons that let you modify the Firefox UI via XUL have to go away. Tree-style tabs[1] for example is one of the major extensions keeping me with FF. There's really no alternative for Chrome since they can't completely change the tab UI like Firefox can. Check out the screenshots!

[1] https://addons.mozilla.org/en-us/firefox/addon/tree-style-ta...


Tree Style Tabs is one of my favourite browser extensions of all time, but it's alreody pretty broken on Firefox 42, and apparently it depends quite a bit on reaching into the browser's UI code and rewritiing chunks of it. I will be very sad if it breaks completely, but not very surprised.


when I lose Tree Style Tabs, I will be deeply saddened and how I browse will be greatly changed.


As is mentioned on the WebExtensions page, they want to add APIs enabling things like that.


Wouldn't something like Tabs Outliner in Chrome be a suitable replacement?

What would be the UX issues there for people switching over? Docking? Tabs styling?


From the Firefox announcement:

Re: Why they are removing XUL:

"XPCOM and XUL are two of the most fundamental technologies to Firefox. The ability to write much of the browser in JavaScript has been a huge advantage for Mozilla. It also makes Firefox far more customizable than other browsers. However, the add-on model that arose naturally from these technologies is extremely permissive. Add-ons have complete access to Firefox’s internal implementation. This lack of modularity leads to many problems.

A permissive add-on model means that we have limited flexibility in changing the foundations of Firefox.

...

"The tight coupling between the browser and its add-ons also creates shorter-term problems for Firefox development. It’s not uncommon for Firefox development to be delayed because of broken add-ons. In the most extreme cases, changes to the formatting of a method in Firefox can trigger problems caused by add-ons that modify our code via regular expressions. Add-ons can also cause Firefox to crash when they use APIs in unexpected ways.

Re: When XUL is being ripped out

Consequently, we have decided to deprecate add-ons that depend on XUL, XPCOM, and XBL. We don’t have a specific timeline for deprecation, but most likely it will take place within 12 to 18 months from now. "

Re: The gap in capability

"A major challenge we face is that many Firefox add-ons cannot possibly be built using either WebExtensions or the SDK as they currently exist. Over the coming year, we will seek feedback from the development community, and will continue to develop and extend the WebExtension API to support as much of the functionality needed by the most popular Firefox extensions as possible."

-------------------------------

It's overly optimistic to assume add-on developers will have new versions ready in just a year if the API they're expected to rewrite everything in isn't even ready yet. I can understand why Mozilla is making this move, but it's being rushed. If WebExtension were ready today then this announcement would be more reasonable, but it's not even close!

Hopefully Mozilla is just trying to scare their add-on developers into action, so they'll speak up and tell Mozilla exactly what they need from WebExtension.

Starting over in a new API means a lot of existing add-ons will probably die anyways, but hopefully the important ones will make the move if Mozilla gives them enough time and support.


> add-ons that modify our code via regular expressions

That's... actually kinda terrifying.


It is. And it's done because Firefox doesn't expose the right hooks and isn't interested in exposing them. They obviously know what's happening already.

From the addon's point of view, the option is do it via regular expressions or not at all; it turns out the most users don't actually care how things get done.


From a technical point of view, what is DownThemAll? It seems like it looks at the structure of the current web page, identifies all links, and then downloads them (in some cases using range requests, as with '90s-era download accelerators), with options for pausing and resuming downloads and renaming them in certain ways.

What prevents this from being done using a Chrome extension to look at the page structure and render some UI, plus a bit of native code using the native messaging API to actually store the files on disk?

https://developer.chrome.com/extensions/nativeMessaging

"Developer frustration" is a more-than-valid reason, but I'm trying to understand if this is a claim that no software like DownThemAll can possibly be written without Mozilla introducing purpose-built extension APIs.


The main reason people install it I thought is because

1. It allows downloads to start and stop. No more 90% of that iso then having to start again.

2. It opens up 4 thingies on the file and downloads simultaneously. So 4 times quicker if the website is restricting bandwidth per connection.


> 1. It allows downloads to start and stop. No more 90% of that iso then having to start again.

I'm surprised Firefox doesn't do this itself, as in, this sounds like a clear bug. Is there some reason the upstream project by itself doesn't get this right?

> 2. It opens up 4 thingies on the file and downloads simultaneously. So 4 times quicker if the website is restricting bandwidth per connection.

... this works in 2015? I'm pretty sure I remember people trying this and website owners deploying countermeasures in the '90s.

Is this secretly a bufferbloat workaround or something?


> this works in 2015? I'm pretty sure I remember people trying this and website owners deploying countermeasures in the '90s.

Oh yes, it works. It works wonders. It can very often quadruple the d/l rate.


> ... this works in 2015?

Sites like rapidshare used to 'mysteriously' fail for me at 95% when running Downthemall.

So yep some 'sites' have/had counter measures.

But if you're not a file sharing site counter measures likes these cost $ to implement.

Not saying it's ethical, like ad block (which also has counter measures) it's a fine like between a power user and taking advantage.


This works pretty well if you are behind a HTTP Proxy and the proxy is limiting speed per connection. This was a common scenario in my university.


With the new APIs, you should be able to have 4 web workers do the downloads in parallel still. (Based on how chrome handles such things)


Browsers tend to have a fairly low cap on simultaneous connections to the same server. Seems to be 6 these days in Chrome or Firefox; I remember 2 from a few years ago.

I'm sorta curious if DownThemAll is using XPCOM to get around any similar good-netizen limits in Firefox.


web workers don't have anything except message passing ability to the main js thread, they cant open sockets, and they can modify the dom, they are nearly worthless.


My understanding is that you can use WebSockets within a web worker. But I haven't done it myself, so that could be wrong. In any event, the service worker specs are all still evolving, and I expect that as people try to build more sophisticated extensions on top of these APIs, they will mature to allow more sophisticated behavior.


Yup, it's basically a spider. The Anticontainer plug-in enhances it to fetch images that are a few steps away from the original page. Looks like it just needs unrestricted XHR, which greasemonkey and every other chrome extension already require. Maybe off-screen rendering, which could look like this: https://www.chromium.org/developers/design-documents/extensi...


> The Anticontainer plug-in enhances it to fetch images that are a few steps away from the original page

You should look at Imagehost Grabber, which is best described as Anticontainer on steroids,

http://imagehost-grabber.eu/index.php


Speaking of Download managers - Chrome still doesn't have a decent one and I for one will miss DownThemAll if it goes away - it is definitely a useful piece of software.

Wonder what the new signing and addon development policies will do to FF market share - there won't be any reason not to just use Chrome anymore. I get the security part but the reason I use FF is because it is less memory hungry and has these extensions that either are not on Chrome or work poorly on it.


I used to use DTA on Firefox but switched a few years ago to native ones. I ended up writing one http://maxelapp.com/ which works for Macs. It can intercept downloads from Chrome with a chrome extension


As people have been saying in the thread that links to the announcement, it is not at all clear that extensions that today can do things in Firefox that are not possible in Chrome will be unable to do those things in the future. The mechanism for things like changing tab management and such would surely change, but that doesn't mean there won't be one.


It also doesn't mean there will be one. Uncertainty about the future of an API is bad, it always causes developers who build on that API to panic and assume the worst.


Uncertainty is bad, but it's unavoidable in this case, for two reasons:

* They need feedback from addon makers in order to design the new API.

* Mozilla does all its work in the open anyhow.

There is no way to avoid an announcement about an intention to change the API, before stating the API in full. Yes, it caused uncertainty, and that's a downside, but open development is generally worth it.


> There is no way to avoid an announcement about an intention to change the API, before stating the API in full. Yes, it caused uncertainty, and that's a downside, but open development is generally worth it.

It's quite possible to announce that they're starting to implement the new API, wait a year or two until it's capable enough, _then_ start talking about deprecating the existing one. Just like it was possible to finish implementing extension signing (or at least to the extent that people can automate signing) before deciding on a time frame to enforce it, or to get the automatic SSL certificate issuer (Let's Encrypt) working before talking about deprecating non-SSL HTTP traffic.

Basically, don't put the cart before the horse, let people implement transition plans. Wanting to have better things is fine; scaring people without having actionable mitigation strategies isn't so nice.


I agree a more organized PR approach might be more effective, but it would be impossible to keep a secret like that when you're doing open development.

People would quickly ask "what is the long-term plan here?" and you can't lie to them.


> People would quickly ask "what is the long-term plan here?" and you can't lie to them.

Sure; but you wouldn't need to. Just don't have a time line set (as opposed to the 12 ~ 18 months set here). They've done it once with Jetpack already, that just didn't go anywhere because the APIs weren't flexible enough to actually not need all the underlying guts. Hopefully this one would have better results, but that requires lots of work on the part of Mozilla to actually implement enough API surface for the thing to be useful.


So you're suggesting they say "we intend to deprecate this feature", but not give a timeline?

It seems much more open and fair to give a timeline. That way it's predictable and lets people plan.


> So you're suggesting they say "we intend to deprecate this feature", but not give a timeline?

Yes; but that only works if they don't actually _have_ a timeline. Do the necessary work to let people explore options, then evaluate and give a timeline (in the same order as what it took to get all that implemented).

The timeline currently is "you have 12 to 18 months, but can't actually start". That's pretty much a recipe for frustration.


I guess it's a matter of opinion, but I greatly prefer it they way they did it.

Announcing "this is going away" without a timeline would make me worry "when? now? in a month?" Instead, by saying "12 to 18 months", I know this is a long-term thing and I can plan for it. The timeline is very useful information.


But the point is that they shouldn't have set the timeline until a replacement was reasonably in place. They could (and should) have said "this is going away 12 to 18 months after we've been able to determine what APIs are needed and implemented them".


With that in mind it's both expected and perfectly fine if addon developers are up in arms about this change, as long as it doesn't stop Mozilla from designing a new API in the open while listening to the developers. Hostile feedback is still feedback.


The concern is basically that they may cover most bases, but will decline to allow some things that could be done today.

Even if they'll support the popular use-cases. What about the fringe uses?


They say they're going to try and cover all the use cases with a new API, but it's clear that that API is in its infancy. Even if it were ready to go today, this would require authors to rewrite their extensions, which will mean that some stable extensions will fall through the cracks.


Not just DTA, but FireGestures, Easy DragToGo, and all the amazing extensions that made me stick by Firefox even though most of my colleagues left for Chrome. It is a sad day.


It's a sad day for me because I use FireGestures entirely for their wheel gesture "[Popup] List all tabs" that allows me to scroll between my tabs from anywhere on the page. It's a strange feature that I've only been able to find in FireGestures (and wish I could find as a stand-alone extension).

I use DTA! with AntiContainer quite frequently.

Palemoon doesn't (properly) support Tab Groups, so I refuse to use it. The add-on is broken and doesn't restore tabs if the browser crashes.

I feel like my only option is to disable FF updates and hope I never get bit by a security issue. As GNU IceCat doesn't work on my machine. Or learn how to compile from source and maintain my own personal fork...

(And before anyone suggests it, using Aurora/Web Dev version is not an option for me. I've already been there, tried that.)


They should spearhead the work on pushing the spec people (and in extension the actual developer teams) at the browser companies to support what is needed to implement DownThemAll in Chrome/Opera/Safari/New Firefox. I have the feeling there isn't that much that is lacking today (sparse file writing support?).

This is clearly a use case that users want.

(DownThemAll is the sole reason I have Firefox installed. Would love to have it work in e.g. Chrome.)


As to NoScript:

- from a linked Mozilla blog post:

> [...] A major challenge we face is that many Firefox add-ons cannot possibly be built using either WebExtensions or the SDK as they currently exist. Over the coming year, we will seek feedback from the development community, and will continue to develop and extend the WebExtension API to support as much of the functionality needed by the most popular Firefox extensions as possible. [...]

(https://blog.mozilla.org/addons/2015/08/21/the-future-of-dev...)

- then, from a link posted at the end of the Mozilla blog post (in an "Update" section):

> [...] One concern people have is that their favorite add-on is no longer going to be supported, especially add-ons for power users. Some of the ones being mentioned are:

> [...] NoScript, [...]

> We’re working with Giorgio Maone, the developer of NoScript, to design the APIs he needs to implement NoScript as a WebExtension. [...]

(https://billmccloskey.wordpress.com/2015/08/21/firefox-add-o...)



Don't be so pessimistic. 18 months are a lot of time, Mozilla folks are smart and often listen to their users, there's the browser.html experiment going on, etc.

I have a gut feeling that in a way or another, things will roughly be the same for extensions developers.


Also, Mozilla are going to have their developers work with add-on developers to help port them.


First, see https://billmccloskey.wordpress.com/2015/08/21/firefox-add-o...

1. these ideas are being announced far in advance of any actual changes

2. firefox devs (see link above) actively want to support current popular extensions, by adding to the Web Extension APIs

In fact, you can participate in this discussion with Mozilla devs more directly: https://webextensions.uservoice.com/forums/315663-webextensi...

Nobody working on Firefox wants to take away your most useful extensions, become a "clone" of Chrome (or any other browser) or otherwise has ulterior motives. The goals is really about improving performance and security and making add-ons easier to write and port between browsers.


Unintended consequences are still consequences.

Most extensions are developed by volunteers; many of the rest are, at best, subsidized as a second job through donations or ad affiliates. These volunteers don't always have the time to fight for the APIs they need.

DownThemAll has been around for a long time. I'm sure its developers are as passionate and involved as you will find anywhere in the Firefox ecosystem. But for the past couple of years, and especially recently, it has felt like Mozilla are working against extension developers rather than for them.

Each developer must decide individually when the sunk cost becomes too much. This move will probably be that point for a number of developers.


Good points, and I'll respond to the "sunk cost" bit by adding that backwards compatibility is a double-edged sword. My favorite pet example of this is MS versus Apple - the former has historically cared deeply about being backwards compatible, while the latter has gone through several hardware and software architecture changes that have required total rewrites from third-party developers.

The situation with Firefox is a little trickier, staying backwards-compatible for web content has always been an important goal, but anything living inside the browser chrome does not get the same assurances.

Right now extensions live somewhere between these two worlds, and we've had over a decade of experience with them so I think it's time to carve out their own space, with dependable APIs that are more stable than the internal browser APIs, but faster-moving and more privileged than the web.


"The new APIs would only allow for a severely limited in functionality, severely stripped down DownThemAll! at best."

This is speculation. The new APIs aren't finished yet, and the announcement they linked to specifically addresses this concern, stating that the new APIs as implemented today don't allow for a lot of existing addons functionality, and specifically states their intention to work with addon developers to ensure that the functionality can be added.


Shame, I used to use it back in college. Like most colleges there was a proxy so torrenting without ssh tunnelling was difficult and I didn't have a server to tunnel in to anyway.

Most people just bought RapidShare (also gone) and downthemall saved mind numbing ctrl+c/v for .rar files along with being able to pause and resume downloads which few others have seemed to grasp as well as they did.


How DTA is this dominant at all is a bit of a mystery to me. It allows me to download sequentially names files in rudimentary batches yes but thats not exactly mindblowing tech.


This is a large reason why:

https://en.wikipedia.org/wiki/First-mover_advantage

But what you listed also isn't all it provides. Imagine being 99% done with a 35.6GB download. If Firefox crashes, you get to start that download from the beginning. As if you hadn't downloaded any part of it at all.

DTA would recover and you'd download 1% of that 35.6GB instead of the entire 35.6GB.

That's a large reason to use DTA over the built-in download manager by itself.


One would hope that anyone providing a 35GB download would provide a torrent or magnet-link download. Maybe I'm wrong, but is seeding the torrent any more expensive than the same bandwidth from a web/ftp server? And then, it's also possible to offload some of the bandwidth onto the swarm, assuming there's more than one concurrent download.

It is a little amazing that the downloading functionality provided out-of-the-box in all the major browsers is such hot garbage still.


So instead of a download manager I need a torrenting program? ;) That's just shifting the problem of "poor downloading" to a different tool. Many would prefer to not have to install another program and would prefer installing an add-on for a program they already have installed.

I do agree with you about torrent/magnet link being provided. Though sometimes large .iso downloads don't come with a torrent/magnet link.

E:

Regardless of file size - even if the download is only 20mb - it's annoying to have to redownload the entire 20mb instead of starting where it left off. What if I'm having connection issues? What if the server is having connection issues? Starting at n% and slowly making progress is better than starting from 0% or having the download be impossible...


> If Firefox crashes, you get to start that download from the beginning.

Pretty sure the built in FF download manager does this too. Either way I had kinda assumed this as a base-line for any downloader. Hell that toy project that I programmed as a kid could continue downloads.

My point is DTA does zero special from my view, except that I know the brand (First mover - as you pointed out) and batch (useful to me as I don't know of any other way). Presumably there is some wget magic that can do it, but I'm on windows so DTA is easier.


At this point, would it make sense to fork Firefox?


I think that's what the Pale Moon (https://www.palemoon.org/) folks were doing some time ago. I'm pretty sure they support most of the original firefox extensions without the signing etc.


I have been using Pale Moon x64 as my daily browser for years now. I have run into a few issues with extension compatibility, (ex: Greasemonkey has issues with versions newer than 1.15.1) but support is generally pretty good. I see it as a viable alternative to Firefox, but it could benefit from a little more mainstream support.


Too bad there's no Mac version.



Would such a thing even be sustainable?

Forks are free. Good, maintained, used forks are another.


There's already GNU IceCat. Pale Moon is quite popular among Windows power users, I've read.


Ridiculous. I've never heard of power users using Pale moon. it seems like the current `hot fad' in comparison to products like SeaMonkey, which power users have always used since the de-coupling of the Mozilla Suite (netscape suite) in 2005. my personal thoughts on pale moon can be found here[0].

[0]: https://www.reddit.com/r/firefox/comments/3ay037/pale_moon_t...


I still use Firefox, but I looked at the status of some of the forks after today's news. It seems that Pale Moon, at the very least, is a true fork, but I don't have a whole lot of confidence that it would be right for me. In particular, it seems that they take a _very_ old-school approach to media on the Web [1], they really only bother with XUL addon compatibility [2], and an attitude that they should only open-source what is legally required of them [3].

There's nothing wrong with these views, but they don't work well for me. I just want my browser to work, with the extensions I like, and until now the way to go about that has been to stick with regular Firefox. Looks like that might not work for very long either.

[1] "Media support is a regularly-discussed topic for Pale Moon users, [...] This is, however, not a goal of Pale Moon because of the inherent shift of focus from document content to media content." https://www.palemoon.org/roadmap.shtml

[2] "If you still like to develop JetPack add-ons for Pale Moon, you will likely have to make a completely separate extension specific for Pale Moon targeting." Ibid.

[3] "There is no requirement or even reason why I should be forced to also release helper applications that are in no way tied to browser operation as Open Source as well." http://forum.palemoon.org/viewtopic.php?f=4&t=7818


Something I just thought of... if these changes are presumably meant to keep malware addons out of the browser, then it's necessarily operating in an infected environment.

(I.e. something already had the ability to do things in the context of the user without that user's permission, and we're just preventing it from doing this one thing via restricting what the user can do)

In what way does this meaningfully secure the browser from malware, with that in mind? If I've got code running in the user's account, I don't need to hook into the browser engine to direct the user at popup ads, phishing sites, harvest their keystrokes, or do any number of other evil things. Hooking into the browser is one of the least interesting things that evil me could be doing. I've got access to the browser's memory, the TCP stream, the ability to launch whatever programs the user does, and am probably traveling along with a payload to allow for privilege escalation.

So, WTF?


The problem these days is established extensions that get legitimately sold or their access data hacked, and malware rolled out to the users.

Or extensions which actually do their job, but after a delay, e.g. a month, deploy the malware payload.

Enforcing someone from Mozilla to take a look at the actual APIs used in the extension is a pretty reasonable way to prevent a lot of this.

Also, disallowing local installs of unsigned stuff is usually a good practice - lots of "download managers" is bundled with "premium" "extensions" turning out to be toolbars collecting and shipping off your data, replacing your ads etc.


> The problem these days is established extensions that get legitimately sold or their access data hacked, and malware rolled out to the users.

Published extensions on addons.mozilla.org (i.e. the built-in extension distribution channel) already go through a review for every update. Someone from Mozilla _already_ has to take a look.

Of course, the addon review queue is currently ~ 10 weeks (according to their blog, from Aug 12). Relative to the 6 week Firefox release cycle, it's like saying if Apple enforced application signing for OSX where you need to submit your app, wait a year and a half, then get a signed version back.


For me, tree style tabs and the ability to have 100s of tabs open is the killer feature of firefox, and the reason I don't use any other browser. I will likely stop updating if these cease to work, and switch to any other browser which allows me to continue my workflow (or maybe start developing my own fork).

I suspect that a substantial portion of firefox's remaining users are there not because of the core browser (which is now almost indistinguishable from chrome), but because of an extension which either no-one cares or no-one is able to port to another browser.


I've started searching for an alternative download accelerator. So far, the best option seems to be the Citrio browser. It's downloading capabilities are on par with dTa: http://www.tutorialspoint.com/articles/citrio-a-chrome-like-...


People still use download managers? I honestly had no idea.


DTA was pretty good for its download acceleration abilities. That's still useful, especially for content hosted far away.


Yeah, I don't doubt it. I've been spoiled by living close to the data I want to download. I imagine that people at the far end of attenuated links would have a very different experience.


Mozilla is dead. Long live Mozilla Extended Support Release! https://www.mozilla.org/en-US/firefox/organizations/


Here's hoping the old extension system deprecation comes after the release of ESR 45, so we can have support for old extensions until March 2017!


Man, if my favourite extensions no longer work properly then I don't know what I'll do.

I think I might have to go full Stallman and wget + email html-only web pages to myself.


Off topic. The url reads like down-the-mall in all lower case. s/Can be a pivot if no backoff from Mozilla/s


Am I the only one who can't read an article when the text is aligned on the left size of a window like that?


Mozilla begin to abuse their power


I'm maintaining two Add-Ons with around 20k active users. I agree with his predictions, but I also find it hard to give to much attention to voices like the one expressed here.

It's hard for me to tell how much of this is genuine concern, or just how many donations or how much income he will loose because of this. I never see numbers from these Add-Ons. The one in question here has 1,300,000 daily active users.

Sure, if I make $1500 per month on an Add-On, I would be very upset if things change. Then I'd rather see the XUL/XPCOM stuff around for another 15 years.


Thanks for implying I'm in it for the money without even knowing me. Nice ad hominem you got there.

Let's just say that the donations do not nearly cover the time I spend developing add-ons, helping other add-on developers out with their questions, volunteering on AMO, writing patches for core Firefox and so on.


I'm sorry if I offended you. All I'm saying is you could be in a position of conflict of interest. Unfortunately, your user base would never mobilize on your behalf, because they don't understand the technical details behind the comprehensive Firefox extension system that would be deprecated.

That's a problem. The user don't understand what they are about to loose, but the big name developers with millions of users have a vested interest in the status quo. I don't have a solution for that. There are a lot of Add-On developers out there for which this is a business. Firefox extensions can generate an income.

At least you make no secret about it. The euro coins are right next to the article. I mean that in a good way.


What does the developer's monetary incentive have to do with the issue of FF deprecating the API? This article would have been just as valid as if it came from a company who made money selling FF extensions.


If users like it enough to spend $1,500 a month on it, it should be a priority for Firefox devs.


This, along with all the other simplifications in tech lately, along with everyone's inexplicable obsession with tablets/phablets/phones, signals the end of the power user era


This is what happens when humans achieve success. Rather than maintain the course they are obligated to exploit to the nth in order to squeeze every gram of return from an idea(product) until they can get no more. Then, they will get creative and pull out every hook & crook to squeeze some more. Thanks Mozilla, for everything up until your mass-collection, feature- bloat, entrapment blitz you've set out upon. Perhaps DTA will make their plug-in compatible with PaleMtoon now. Chances are the forks will come out on top & Moz can go f-up Thunderbird & Seamonkey for awhil as their top-heavy pyramid topples.

edit:admittedly a pipe dream, markets ruled by duops and oligops never lose the top players, they just cannibalize each othet and bloat.


I don't understand what is being exploited?


The end-user, of course.

"Surreptitious spying is all the rage. Everybody is doing it."




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: