ISO hoster wanted

[Stefan Reich]

Hi guys -

I'm looking for a web hoster that can handle ISOs. Total size 3 GB or
so. (Right now, 1 GB will suffice.)

SourceForge might do it through their FRS (file release system), but
that is REALLY buggy in my experience. If an upload is interrupted,
the mirrors will mirror the broken file - and after that, not accept
any corrected versions. Had that a couple times and stopped using FRS
altogether.

I managed to upload it to SourceForge web, but they actually have a
size limit in the web server and tell you to use FRS instead.

So, to cut a long story short - are there any hosting options out
there that we can use?

(Oh, and yes, this means that I am making Lua OS ISOs. They already
work. Live and with HD install! =))

Cheers,
Stefan

[Christian Thaeter]

Am Tue, 6 Sep 2011 19:41:04 +0200
schrieb Stefan Reich <[hidden email]>:

> Hi guys -
>
> I'm looking for a web hoster that can handle ISOs. Total size 3 GB or
> so. (Right now, 1 GB will suffice.)

How about bittorrent? I could help then seeding with my server

>
> SourceForge might do it through their FRS (file release system), but
> that is REALLY buggy in my experience. If an upload is interrupted,
> the mirrors will mirror the broken file - and after that, not accept
> any corrected versions. Had that a couple times and stopped using FRS
> altogether.
>
> I managed to upload it to SourceForge web, but they actually have a
> size limit in the web server and tell you to use FRS instead.
>
> So, to cut a long story short - are there any hosting options out
> there that we can use?
>
> (Oh, and yes, this means that I am making Lua OS ISOs. They already
> work. Live and with HD install! =))
>
> Cheers,
> Stefan
>

[Stefan Reich]

Ah. You would seed it then? Sounds great. I'm planning to switch to
Bittorrent at some point anyway.

The only question is, how do we get the file from me to you in the
first place. Oh, if you have a server, I could upload it, right?

This does seem nice. Please tell me if you're inclined to proceed with
that plan. And we should continue this in private I assume :)

Cheers,
Stefan

On Tue, Sep 6, 2011 at 11:01 PM, Christian Thaeter <[hidden email]> wrote:
> How about bittorrent? I could help then seeding with my server

[Ezra Sims]

All you would need to do is host the torrent and provide a link via the mailing list. I'm sure there's a few people reading who wouldn't mind seeding for you.

Finding trackers for it would be the only marginally difficult part. I'd probably start with the Pirate Bay myself - Image aside, they're very stable and mirrored/duped by quite a few sites..

On Tue, Sep 6, 2011 at 8:13 PM, Stefan Reich <[hidden email]> wrote:

Ah. You would seed it then? Sounds great. I'm planning to switch to
Bittorrent at some point anyway.

The only question is, how do we get the file from me to you in the
first place. Oh, if you have a server, I could upload it, right?

This does seem nice. Please tell me if you're inclined to proceed with
that plan. And we should continue this in private I assume :)

Cheers,
Stefan

On Tue, Sep 6, 2011 at 11:01 PM, Christian Thaeter <[hidden email]> wrote:
> How about bittorrent? I could help then seeding with my server

[Christian Thaeter]

Am Tue, 6 Sep 2011 21:10:18 -0400
schrieb Ezra Sims <[hidden email]>:

> All you would need to do is host the torrent and provide a link via
> the mailing list. I'm sure there's a few people reading who wouldn't
> mind seeding for you.
>
> Finding trackers for it would be the only marginally difficult part.
> I'd probably start with the Pirate Bay myself - Image aside, they're
> very stable and mirrored/duped by quite a few sites..

It doesn't need trackers anymore or? You just create the torrent and
host it (from slow home) and then the swarm will distribute.

>
> On Tue, Sep 6, 2011 at 8:13 PM, Stefan Reich <
> [hidden email]> wrote:
>
> > Ah. You would seed it then? Sounds great. I'm planning to switch to
> > Bittorrent at some point anyway.
> >
> > The only question is, how do we get the file from me to you in the
> > first place. Oh, if you have a server, I could upload it, right?

Bittorrent :) .. if your bandwidth permits (if only for a initial
seeding phase). I am possibly going to setup some way to rsync torrents
and files to our server, but for security/legality reasons this needs
either password or human review then and will take some time to setup.

Btw Its not the case that I have a massive server, just a humble
rootserver (hetzner/germany) with iirc 1TB Traffic limit. I count on as
many people as possible to help here, wasn't that the idea with
bittorrent?

        Christian

> >
> > This does seem nice. Please tell me if you're inclined to proceed
> > with that plan. And we should continue this in private I assume :)
> >
> > Cheers,
> > Stefan
> >
> > On Tue, Sep 6, 2011 at 11:01 PM, Christian Thaeter <[hidden email]>
> > wrote:
> > > How about bittorrent? I could help then seeding with my server
> >
> >

[Ezra Sims]

On Tue, Sep 6, 2011 at 9:45 PM, Christian Thaeter <[hidden email]> wrote:

Am Tue, 6 Sep 2011 21:10:18 -0400
schrieb Ezra Sims <[hidden email]>:

> All you would need to do is host the torrent and provide a link via
> the mailing list. I'm sure there's a few people reading who wouldn't
> mind seeding for you.
>
> Finding trackers for it would be the only marginally difficult part.
> I'd probably start with the Pirate Bay myself - Image aside, they're
> very stable and mirrored/duped by quite a few sites..

It doesn't need trackers anymore or? You just create the torrent and
host it (from slow home) and then the swarm will distribute.

Yes, DHT (distributed hash table) makes trackerless torrents easy to do. Trackers are preferred mainly for visibility reasons (read: getting the image out to people who wouldn't otherwise be looking for it). They do have the nice advantage of ensuring that every possible peer and seed is connected to each other.

[Gregory Bonik]

Ezra Sims wrote:
> Finding trackers for it would be the only marginally difficult part.

Maybe http://openbittorrent.com/ can help.

--Gregory

[Axel Kittenberger]

> Bittorrent :) .. if your bandwidth permits (if only for a initial
> seeding phase).

Amazon S3 can be used to seed dirctly if you've some cents to spare each month
http://carltonbale.com/how-to-create-and-seed-a-torrent-download-on-amazon-s3

[Dimiter 'malkia' Stanev]

What exactly do you have, that your iso is 3GB? The lua device and
printer drivers?

[Christian Thaeter]

Am Wed, 07 Sep 2011 18:17:21 +0300
schrieb "Dimiter \"malkia\" Stanev" <[hidden email]>:

> What exactly do you have, that your iso is 3GB? The lua device and
> printer drivers?
>
>

Don't forget to compress it, when you don't use a
compressed Filesystem then users will thank you.

        Christian

[Rena]

On Wed, Sep 7, 2011 at 09:17, Dimiter "malkia" Stanev <[hidden email]> wrote:
> What exactly do you have, that your iso is 3GB? The lua device and printer
> drivers?
>
>
>

I can imagine printer drivers taking up 3GB... ;-)

--
Sent from my toaster.

[Stefan Reich]

Haha :)) Nooo. This is a misunderstanding!

I was projecting 3 GB of server space to have space for multiple
versions (archives).

The Lua OS 0.9 CD image is 507 MB (download here, incidentally:
http://sourceforge.net/projects/safelua/files/isos/).

It's a remastered Knoppix plus Java, Lua and Lua OS. (Lua OS itself is
a 2 MB package...)

For some reason, if you remaster Knoppix without any changes, it seems
to turn out a lot larger than the original Knoppix CD (>900 MB).
Something is probably different in the compression they use or
something.

So I had to delete a lot of stuff to arrive around the 500 MB mark. I
am sure this can be slimmed down a lot further.

We might want to keep some stuff anyway - printing from Lua OS would
be nice (once it can do that). We also want drivers for as much
hardware as possible.

So if anyone is inclined to slim the system some more, here's a guide
on how to master a Lua OS CD: http://luaos.net/pages/mastering.php

I got the Knoppix remastering process simplified a lot, I'm really
happy with that :) Anyone with some basic Unix knowledge can do it
now.

So the next release might even be quite a bit smaller than those 507
MB. I don't care that much, but it would certainly simplify uploading
and spreading new releases.

Cheers,
Stefan


On Wed, Sep 7, 2011 at 5:17 PM, Dimiter "malkia" Stanev
<[hidden email]> wrote:
> What exactly do you have, that your iso is 3GB? The lua device and printer
> drivers?

[Stefan Reich]

> A basic linux install can come in at under 2MB. I cannot imagine
> adding lua, java + X server or similar would increase it beyond even
> 100MB....

Well - do you only think (=dream) that or have you really checked?
You're welcome to produce a working Linux in 100 MB if you think that
is possible. Things might be a lot larger than you think, especially
if you use recent software. All the "mini" distros I tried  are either
extremely outdated (DSL) or kinda suck in various ways. so I went with
Knoppix which actually works and has a good feel to it, overall.

Here's a list of the largest packages in the Lua OS CD (uncompressed size in K):

linux-image-2.6.39.3                               81202
openjdk-6-jre-headless                             78224
chromium                                           67584
libgl1-mesa-dri                                   49960
vim-runtime                                       25416
libgtk2.0-common                                   21484
libflite1                                         19816
xulrunner-1.9.1                                   19092
libc6-dev                                         18536
libicu44                                           18336
perl-modules                                       15856
gcc-4.6                                           15412
chromium-l10n                                     15028
libgs8                                             14672
gnome-icon-theme                                   13876
binutils                                           13660
perl                                               13172

So yeah. The Linux image alone is 81 MB. I left in Chrome because it
sucks not to have a browser IMO. And a Lua browser is still some way
off... :p

There are a couple of obvious improvement candidates of course.
vim-runtime may surely be replaced by good old standard vi (does that
still exist? :)).

Well then. Feel free to master your own CD. The size race is open!

-Stefan

On Thu, Sep 8, 2011 at 7:21 AM, Daurnimator <[hidden email]> wrote:

> On 7 September 2011 03:41, Stefan Reich
> <[hidden email]> wrote:
>> Hi guys -
>>
>> I'm looking for a web hoster that can handle ISOs. Total size 3 GB or
>> so. (Right now, 1 GB will suffice.)
>>
>> SourceForge might do it through their FRS (file release system), but
>> that is REALLY buggy in my experience. If an upload is interrupted,
>> the mirrors will mirror the broken file - and after that, not accept
>> any corrected versions. Had that a couple times and stopped using FRS
>> altogether.
>>
>> I managed to upload it to SourceForge web, but they actually have a
>> size limit in the web server and tell you to use FRS instead.
>>
>> So, to cut a long story short - are there any hosting options out
>> there that we can use?
>>
>> (Oh, and yes, this means that I am making Lua OS ISOs. They already
>> work. Live and with HD install! =))
>>
>> Cheers,
>> Stefan
>>
>>
>
> I'm very interested in how you have such a big image already....
> A basic linux install can come in at under 2MB. I cannot imagine
> adding lua, java + X server or similar would increase it beyond even
> 100MB....
>
> D
>

[Axel Kittenberger]

I worked in a company which developed embedded-linux devices for which
a software image fitted into a 2MB EPROM. Back then we had a 4MB
EProms, and for a safe update always used only half, so the old image
is kept while a new image been written. 2MB was compressed: Linux
kernel with all needed drivers, clibs and other essential libraries,
busybox, some network tools, even a webserver, and of course the
custom applications been developed.

I wonder what audience your product is targeting?

[Stefan Reich]

> I wonder what audience your product is targeting?

Well - not embedded systems, at this point. Regular PCs.

I wonder - did your Linux distros include an X server?

I mean, I'm not against small distros. Why would I be. I'd love to
have a small distro :) I just did something that works. It may be
improved upon in any way fit.

As I said - the size race is open!

Greetings,
Stefan


On Thu, Sep 8, 2011 at 6:25 PM, Axel Kittenberger <[hidden email]> wrote:

> I worked in a company which developed embedded-linux devices for which
> a software image fitted into a 2MB EPROM. Back then we had a 4MB
> EProms, and for a safe update always used only half, so the old image
> is kept while a new image been written. 2MB was compressed: Linux
> kernel with all needed drivers, clibs and other essential libraries,
> busybox, some network tools, even a webserver, and of course the
> custom applications been developed.
>
> I wonder what audience your product is targeting?
>
>

[steve donovan]

On Thu, Sep 8, 2011 at 6:25 PM, Axel Kittenberger <[hidden email]> wrote:
> I wonder what audience your product is targeting?

Good question. And why is Perl along for the ride ?  That's one heavy camel!

Tom's Root disk was a little Linux distro that fitted on those old
fashioned irritating things we used to slide into our computers (about
1.2 meg).  I mention it because apart from Busybox he replaced a lot
of userland code with Lua scripts.

What you need (almost) is something like Android: Linux core + Java.

steve d.

[Stefan Reich]

On Wed, Sep 7, 2011 at 6:23 PM, Christian Thaeter <[hidden email]> wrote:
> Don't forget to compress it, when you don't use a
> compressed Filesystem then users will thank you.

I use what I believe is the standard compressed FS in Knoppix
(create_compressed_fs with gzip). Maybe the Knoppix makers use some
other magic option though, dunno. I tried the 7zip option, but it
operates at a speed of about 50k/s (yes, really), so I skipped that.
:]

[Stefan Reich]

On Thu, Sep 8, 2011 at 6:41 PM, steve donovan <[hidden email]> wrote:
> On Thu, Sep 8, 2011 at 6:25 PM, Axel Kittenberger <[hidden email]> wrote:
>> I wonder what audience your product is targeting?
>
> Good question. And why is Perl along for the ride ?  That's one heavy camel!

Because it's probably used in lots of places in Knoppix? I'd like to
believe so. Maybe it's not. I'll freely admit I'm not what you'd call
a Linux expert. :)

Anyways, the size issue is not THAT a big thing after all, is it? The
installed Lua OS 0.9 fits on a 4 GB partition easily. Even my old
notebook can handle that.

And I'm sure we'll be able to slim this down further. I'm still
waiting for the experts to jump in instead of just demanding stuff. :p

Stefan

[Jorge Visca]

On jue, 2011-09-08 at 18:33 +0200, Stefan Reich wrote:
> > I wonder what audience your product is targeting?
>
> Well - not embedded systems, at this point. Regular PCs.
>
> I wonder - did your Linux distros include an X server?
>
> I mean, I'm not against small distros. Why would I be. I'd love to
> have a small distro :) I just did something that works. It may be
> improved upon in any way fit.

You could base it from OpenWRT. A full fledged Linux distro that fits in
16Mb flash/32 Mb RAM with space to spare, and supports several embedded
platforms, besides PCs. Comes with Lua. And Xserver if you're in that
kind of stuff.

[Christian Thaeter]

Am Thu, 8 Sep 2011 18:44:32 +0200
schrieb Stefan Reich <[hidden email]>:

> On Wed, Sep 7, 2011 at 6:23 PM, Christian Thaeter <[hidden email]>
> wrote:
> > Don't forget to compress it, when you don't use a
> > compressed Filesystem then users will thank you.
>
> I use what I believe is the standard compressed FS in Knoppix
> (create_compressed_fs with gzip). Maybe the Knoppix makers use some
> other magic option though, dunno. I tried the 7zip option, but it
> operates at a speed of about 50k/s (yes, really), so I skipped that.
> :]
>

squashfs is very cool when you do readonly media, not only it
compresses well, but its also very fast and compresses filesystem
metadata and and iirc deduplicates data, it also uses parallel
compressing/decompressing on multi core cpus. I would use it hands
down :)

Otherwise for knoppix I once experimented with hardlinking all same
files, but this broke the 'installabiliy' of knoppix (one has to break
the right links when installing), I propose a schema doing this
(keeping the list whats got hardlinked and how so). I don't know if that
found the way into knoppix mainstream, anyways possibly there are some
(but not all) files hardlinked and when you remaster in the wrong way
that may well yield such a size increase (back in time my complete
hardlinking saved quite some space, thats years ago, cant remember how
much but possibly in the 15%+ Range). Meanwhile 'fdupes' has support
for such hardlinking but you have to be sure that your whole toolchain
takes care.

        Christian