DAS-BAHN-FORUM.de

Herzlich willkommen im ersten offiziellen Forum zum Programm BAHN von JBSS
Aktuelle Zeit: Mittwoch 18. September 2019, 17:29

Alle Zeiten sind UTC + 1 Stunde




Ein neues Thema erstellen Auf das Thema antworten  [ 74 Beiträge ]  Gehe zu Seite Vorherige  1, 2, 3, 4, 5  Nächste
Autor Nachricht
 Betreff des Beitrags: Re: Generating Bahn Layouts
BeitragVerfasst: Samstag 27. März 2010, 21:55 
Offline

Registriert: Donnerstag 8. Mai 2008, 22:56
Beiträge: 27
Wohnort: Nederland
Hello everyone

Due to some progress at Openstreetmap (they create another way off accessing Openstreetmap data (in a compressed way) and some bug resolving I've created last 2 weeks version 0.19.
This version is available as from now.

So take a look at:

http://www.xs4all.nl/~wanl0049/ops2jbss/

again.


It solves also the issue where Chris is struggling with and it is much faster.

One small issue still appears (the 'cache' are is not cleaned after everything is well processed).

Please use the CCA possibility in the menu, to clean this.

Regards, Bert Tijhuis


Nach oben
 Profil  
Mit Zitat antworten  
 Betreff des Beitrags: Re: Generating Bahn Layouts
BeitragVerfasst: Mittwoch 26. Mai 2010, 21:16 
Offline

Registriert: Donnerstag 24. Mai 2007, 02:26
Beiträge: 170
I am getting an error: "XMLfile GB.OSM is invalid and will be deleted (Please restart the extractor again)" gb.osm in this case was the file from http://download.geofabrik.de/osm/europe/ which I am assured is valid. It turns out that the problem was converting the supplied filename to upper case. There's no need to do this: on a case-sensitive filesystem you will not find the file, and on a case-insensitive filesystem it doesn't matter what case you supply.


Nach oben
 Profil  
Mit Zitat antworten  
 Betreff des Beitrags: Re: Generating Bahn Layouts
BeitragVerfasst: Donnerstag 27. Mai 2010, 20:31 
Offline

Registriert: Donnerstag 8. Mai 2008, 22:56
Beiträge: 27
Wohnort: Nederland
Chris hat geschrieben:
I am getting an error: "XMLfile GB.OSM is invalid and will be deleted (Please restart the extractor again)" gb.osm in this case was the file from http://download.geofabrik.de/osm/europe/ which I am assured is valid. It turns out that the problem was converting the supplied filename to upper case. There's no need to do this: on a case-sensitive filesystem you will not find the file, and on a case-insensitive filesystem it doesn't matter what case you supply.


I wasn't aware of this nice GEOFABRIK.
In the meantime I will download a file a try to find why it is failing
Regards,


Nach oben
 Profil  
Mit Zitat antworten  
 Betreff des Beitrags: Re: Generating Bahn Layouts
BeitragVerfasst: Donnerstag 27. Mai 2010, 22:31 
Offline

Registriert: Donnerstag 24. Mai 2007, 02:26
Beiträge: 170
My Perl is about as good as my Swahili, but there are two problems here. First the casing:
Code:
# ops2jbss.pl:647
                $veld2 =~ tr/a-z/A-Z/;

I don't know why this line is here, since the few places I've see that handle user input are already rigged for both cases, e.g.
Code:
                if ($choise =~ /^[gG]$/) { # both cases specified
                if ($choise =~ /^cca/i) { # case-insensitive match


The error message comes from here:
Code:
# ops2jbss_e.pl:170
  print "Process file: $file\n" if ($debug);
  $dbh->do("begin transaction immediate");
  my $ref = eval { XMLin($file); };
  if ($@) {
      print "XMLfile $file is invalid and will be deleted (Please restart the extractor again)\n" if ($debug);
      unlink($file);
      return 0;
   }

That looks like it's producing that same message regardless of what the error actually was, which IMO makes it poor to assume it'll be an invalid file, and rather nasty to insist on unlinking the file afterwards.

Just my .02


Nach oben
 Profil  
Mit Zitat antworten  
 Betreff des Beitrags: Re: Generating Bahn Layouts
BeitragVerfasst: Freitag 28. Mai 2010, 18:55 
Offline

Registriert: Donnerstag 8. Mai 2008, 22:56
Beiträge: 27
Wohnort: Nederland
Hello Chris,

Thanks for digging through the code:

The real problem is:\
Choise ?: > o great_britain.osm
Process file: GREAT_BRITAIN.OSM
Out of memory!
Out of memory!
Out of memory!

Because these OSM file are realy large (I've tested it with a file of 3 GBytes)

Due to the fact that XMLin is part of the XML::Simple module (not maintained by myself) I can't solve this issue. Try to use smaller files


But please in you're case download the newest sources,
These won't delete the file afterwards, and specified the root course of the failure.

I've noticed that you are still working with an older release please use release 0.20 or upwards

Regards,
Bert Tijhuis


Nach oben
 Profil  
Mit Zitat antworten  
 Betreff des Beitrags: Re: Generating Bahn Layouts
BeitragVerfasst: Freitag 28. Mai 2010, 19:18 
Offline

Registriert: Donnerstag 24. Mai 2007, 02:26
Beiträge: 170
BertTijhuis hat geschrieben:
Because these OSM file are realy large (I've tested it with a file of 3 GBytes)

That was not the cause of the issues I raised. The issue I raised was that you are incorrectly converting the case of the filename, and then throwing back an error message which is inaccurate. This is independent of the file being impossible to process this way on a 32-bit machine.
Code:
chris@testbox:~/ops2jbss$ ls -l test.osm
-rw-r--r-- 1 chris chris 0 2010-05-28 19:22 test.osm
[...]
Choise ?:  > o test.osm
Process file: TEST.OSM
XMLfile TEST.OSM is invalid (Please restart the extractor when nescessarely)
Reason:
File does not exist: TEST.OSM at ops2jbss_e.pl line 187


Zitat:
I've noticed that you are still working with an older release please use release 0.20 or upwards

I was using 0.19 yesterday, and you released 0.20 today. I notice that 0.20 still has the upper-case tr - why is it there?

Also, what release should I have been using yesterday? ;)


Nach oben
 Profil  
Mit Zitat antworten  
 Betreff des Beitrags: Re: Generating Bahn Layouts
BeitragVerfasst: Samstag 29. Mai 2010, 02:52 
Offline
Benutzeravatar

Registriert: Freitag 18. Februar 2005, 12:50
Beiträge: 1858
Wohnort: Marbach am Neckar
You can reduce file size with the tool osmosis: you can create a much smaller xml-file containing only railway lines.

_________________
Bild


Nach oben
 Profil  
Mit Zitat antworten  
 Betreff des Beitrags: Re: Generating Bahn Layouts
BeitragVerfasst: Sonntag 13. Juni 2010, 10:10 
Offline

Registriert: Donnerstag 8. Mai 2008, 22:56
Beiträge: 27
Wohnort: Nederland
Chris hat geschrieben:
BertTijhuis hat geschrieben:
Because these OSM file are realy large (I've tested it with a file of 3 GBytes)

That was not the cause of the issues I raised. The issue I raised was that you are incorrectly converting the case of the filename, and then throwing back an error message which is inaccurate. This is independent of the file being impossible to process this way on a 32-bit machine.
Code:
chris@testbox:~/ops2jbss$ ls -l test.osm
-rw-r--r-- 1 chris chris 0 2010-05-28 19:22 test.osm
[...]
Choise ?:  > o test.osm
Process file: TEST.OSM
XMLfile TEST.OSM is invalid (Please restart the extractor when nescessarely)
Reason:
File does not exist: TEST.OSM at ops2jbss_e.pl line 187


Zitat:
I've noticed that you are still working with an older release please use release 0.20 or upwards

I was using 0.19 yesterday, and you released 0.20 today. I notice that 0.20 still has the upper-case tr - why is it there?

Also, what release should I have been using yesterday? ;)



-----
I've now 2 answers to you're questions

1) I will fix the bug that only occurs in a none Windows environment. (within 2 months)

2) I will start to work on a version that could use precompiled OSM files from the internet (e.g. http://download.geofabrik.de/osm/europe/ )
and use the splitter that is also used for the MKgmap utility. (within 2 months).



Of course in the mean time you can use OSMOSIS and rename you're filenames to lower case.


Regards,
Bert Tijhuis


Nach oben
 Profil  
Mit Zitat antworten  
 Betreff des Beitrags: Re: Generating Bahn Layouts
BeitragVerfasst: Donnerstag 5. August 2010, 20:30 
Offline
Benutzeravatar

Registriert: Freitag 18. Februar 2005, 12:50
Beiträge: 1858
Wohnort: Marbach am Neckar
I've cut of the German postings and moved them to "Probleme und Fehlermeldungen".

_________________
Bild


Nach oben
 Profil  
Mit Zitat antworten  
 Betreff des Beitrags: Re: Generating Bahn Layouts
BeitragVerfasst: Samstag 7. August 2010, 05:51 
Offline

Registriert: Donnerstag 24. Mai 2007, 02:26
Beiträge: 170
Something seems to be out of order somewhere. Did my first live extraction with 0.20 and got the following:
Zitat:
DBD::SQLite::db do failed: SQL logic error or missing database
cannot start a transaction within a transaction at ops2jbss_e.pl line 199.

It comes at the point where "$cache-file will be processed". Centred at 51.74N 4.065W, 50x50km, scale 64.


Nach oben
 Profil  
Mit Zitat antworten  
 Betreff des Beitrags: Re: Generating Bahn Layouts
BeitragVerfasst: Freitag 14. Januar 2011, 19:30 
Offline

Registriert: Donnerstag 24. Mai 2007, 02:26
Beiträge: 170
More bugs:

1. Start with no database. Attempt to import a file. You get a zero-length file where the database should be, and the SQL process bails with "table 'way' does not exist" (etc.). The database has not been initialised correctly. CDB does not rectify this (since all it does is unlink the file).
Workaround: start an online extraction, and abort shortly after it starts - the database is then initialised and the import can be performed.

2. Program deletes files it has attempted to import, even when the import has failed due to #1.


Nach oben
 Profil  
Mit Zitat antworten  
 Betreff des Beitrags: Re: Generating Bahn Layouts
BeitragVerfasst: Sonntag 20. November 2011, 15:48 
Offline

Registriert: Sonntag 20. November 2011, 15:37
Beiträge: 4
Anything yet on this?


Nach oben
 Profil  
Mit Zitat antworten  
 Betreff des Beitrags: Re: Generating Bahn Layouts
BeitragVerfasst: Dienstag 22. November 2011, 21:52 
Offline

Registriert: Donnerstag 8. Mai 2008, 22:56
Beiträge: 27
Wohnort: Nederland
Hello,

Just after one year, there is some news about this.

Due to a remark of Hamzaa I've found out that the current procedure isn't working anymore correctly.

So I've updated the version (021) and this is available as of 20 november 2011.

@Chris: With this current version it is possible to process large files from www.geofabrik.de but of course you've to unzip these files first (7zip)
The program doesn't consume that much memory anymore and you don't run out of memory.

Due to the fact that we don't run out of memory anymore I've introduced a disadvantage that its running slower than the previous versions.

Please try this new version, and give me some feedback


Nach oben
 Profil  
Mit Zitat antworten  
 Betreff des Beitrags: Re: Generating Bahn Layouts
BeitragVerfasst: Freitag 25. November 2011, 18:46 
Offline

Registriert: Donnerstag 8. Mai 2008, 22:56
Beiträge: 27
Wohnort: Nederland
It seems that there is something wrong in Generating Bahn layouts.

Partly this has to do with the richness of the Openstreetmap data. So you cant find the needle in the haystack anymore.
And sometimes some large rectancular lines are generated.

So I still working on this issue and hope to solve this asap

Regards :o


Nach oben
 Profil  
Mit Zitat antworten  
 Betreff des Beitrags: Re: Generating Bahn Layouts
BeitragVerfasst: Freitag 25. November 2011, 23:33 
Offline

Registriert: Donnerstag 8. Mai 2008, 22:56
Beiträge: 27
Wohnort: Nederland
Hello,

Just before going to Kuala Lumpur I've finalized this version 0.22. (This is a bugfix of 0.21)
Please DONt use version 0.21

Tested with several area's (from Geofabrik.de) with Luxembourg (Pretty small country and I like this), Ile de France (on request) and Kuala Lumpur and ofcourse the Netherlands

Everything seems to be working correctly

:oops: :oops: :mrgreen:


Nach oben
 Profil  
Mit Zitat antworten  
Beiträge der letzten Zeit anzeigen:  Sortiere nach  
Ein neues Thema erstellen Auf das Thema antworten  [ 74 Beiträge ]  Gehe zu Seite Vorherige  1, 2, 3, 4, 5  Nächste

Alle Zeiten sind UTC + 1 Stunde


Wer ist online?

Mitglieder in diesem Forum: 0 Mitglieder und 2 Gäste


Sie dürfen keine neuen Themen in diesem Forum erstellen.
Sie dürfen keine Antworten zu Themen in diesem Forum erstellen.
Sie dürfen Ihre Beiträge in diesem Forum nicht ändern.
Sie dürfen Ihre Beiträge in diesem Forum nicht löschen.
Sie dürfen keine Dateianhänge in diesem Forum erstellen.

Suche nach:
Gehe zu:  
cron

Bei allen Fragen zum Forum wenden Sie sich bitte an ronny@das-bahn-forum.de!

Unterstützen Sie das BAHN-Forum mit einer Bestellung bei Amazon.de!
Unterstützt durch phpBB
Version des BAHN-Forums: 4.5.1

Mit dem Lesen und Verfassen von Beiträgen akzeptieren Sie die Regelungen zur Nutzung des BAHN-Forums!