Access Blocked while using OSM Classic

MandM · October 08, 2011, 11:21:56

Hi, I'm from the (german) OSM community and wrote the quoted summary.

Yes it might be possible that the tone of the admins is sometimes rough (even to me as a member) but on the other hand I know that they are under high pressure. The problem is, that the universities, that currently host OSM for free will kick off the project, because their network hardware is to much stressed (connections and traffic). For that reasons we had to ban the most heavy used apps to make sure that the project (that creates a free database not deploys a webmap with similar performance than google maps) is still alive.

Of course this is annoying to endusers but on the other hand this tools doesn't fullfil the Tile usage policy and thats not fair for others and esp. to the volunteeres of the project who just want to do a good job
http://wiki.openstreetmap.org/wiki/Tile_usage_policy

So what are possible solutions (as pointed out above)?
-use another tiles hoster as Cloudmade or Mapquest http://wiki.openstreetmap.org/wiki/TMS
- buy a service account by a third party company http://wiki.openstreetmap.org/wiki/List ... Consulting
- setup your own rendering stack
- use only vector maps and render it on device

At this point donations won't fix the problem itself (max. connections in university hardware), but feel free to join the discussions here: http://www.openstreetmap.org/user/!i!/diary/15046

Menion · October 08, 2011, 14:03:08

Hello MandM,
thanks for info. I'm not registered as part of german community and I don't personally speak german but please say them this (and this is also to others)

current (already done) steps:
--------------------
1. today I'll release new version 1.3.4
2. direct downloading from openstreemap.org server will be disable. I'll leave only viewing and caching data. This should reduce locus to only some mobile viewer
3. today I finished "new" vector maps, (update that finally cover whole planet) so I hope that many users will use them
4. I also moved MapQuest server on first place in list of maps, this should also help to reduce using of OSM classic server
5. And finally I donated amount that at least should cover some damage done due to enormous numbers of users

plans:
------------
1. I was yesterday playing for whole day with mapnik, openlayers, mod_tile and some other stuff and even when I never did something like this, seems it should work (I have already simple map of Czech republic in Locus from own server ... nice), so I decided that I'll create own tile server that will move completely all trafic from osm classic server to this (only for locus) server. One my friend is more experience with these server stuff so I'll meet him in monday and hope that we should start work on this soon ...

that's all

MandM · October 08, 2011, 15:53:51

I added it to the forums (that are within the general OSM forums and so you can logon with your OSM account)

Khaytsus · October 08, 2011, 17:19:06

I've tried to figure out what an Amazon EC2 server would cost, but its calculator is using all terms I don't quite understand, even as a typical systems administrator.. I'd have to look more into what all of it means.

But I wonder if EC2 + http://tiledrawer.com/ is a possible alternative? Unless I have the calculator very wrong, I'm seeing something like $20-$40 a month, but again, I'm mostly guessing.

Anyone here use EC2 and can estimate better?

MandM · October 08, 2011, 17:26:02

AFAIK there are some folks doing rendering on EC2 but currently don't know who that was...

Menion · October 08, 2011, 17:48:22

you really mean 20-40$? Seems like nonsence ... by my simple compute it will be around 500$ per month. There is really a lot of users

. I'll try some small server to test how this all will works, but EC2 seems to be too expensive solution ...

Khaytsus · October 08, 2011, 22:30:12

Quote from: "menion"you really mean 20-40$? Seems like nonsence ... by my simple compute it will be around 500$ per month. There is really a lot of users . I'll try some small server to test how this all will works, but EC2 seems to be too expensive solution ...

See, first off the calculator was a little confusing, and I have no idea how much actual data will be used

I've looked around a little into how to set up my own tile server but I'd need to talk to the owner of the boxen I use.. Big machines on fat pipes, but not sure he'd want that sort of commitment.

That said, any idea how much data Locus users would download in an average month, order of magnitude... 10g. 50g.. 200g?

Menion · October 09, 2011, 07:38:49

during september was downloaded around 1.2TB of vector maps
as I calculate, I expect around 2TB of online maps when there will be limit to 10k tiles per day. So quite a lot ...

Khaytsus · October 09, 2011, 15:32:35

Quote from: "menion"during september was downloaded around 1.2TB of vector maps
as I calculate, I expect around 2TB of online maps when there will be limit to 10k tiles per day. So quite a lot ...

Yikes, much higher than I expected. Of course, 10k tiles per day is much higher than I expected too.

MandM · October 21, 2011, 09:18:28

Yes, so you see whats the problem if we have 10 of this tools on our servers

BTW Currently there is no feedback by the OSM Admins if Locus can be unblocked. I asked them now again...

MandM · October 29, 2011, 22:00:21

You might be interested in the visualisation and compare
http://www.openstreetmap.org/user/!i!/diary/15190

Menion · October 29, 2011, 22:51:56

thanks, I already got echo from jusc here viewtopic.php?f=10&t=1229#p7296 . Pity that no one from German styled OSM maps or Classic openstreetmap.org not communicate. Have to write more emails at Monday. I sure that today next very unpopular update will make them happy

(limited already limited downloading). At least someone ... because I'm going to take some pills on head from this all ... (don't worry, I hate pills) ...

MandM · October 29, 2011, 23:31:51

Indeed I'm registret for the german mails, too. But we don't promote the tiles as we don't have that much resources.

pnorman · November 21, 2011, 10:13:04

I did some calculations on IRC based on munin stats for konqui and albi, the two tile proxies. Based on 23 TB/month I estimated it at about 2500-3500/month to do with EC2 what the OSM tile servers do. About 1k/month for an instance, the remainder for bandwidth.

EC2 isn't a great option. It's good if you want to be able to rapidly scale up some types of services, but serving tiles isn't one of them. EC2 costs about $100/TB

Someone pointed to Hetzner as a cheaper option, being 6.9 euros/TB.