« zpět na výpis měsíce |

[Talk-cz] Import skript z uir_adr (fwd)

Vlákno 27.8. - 27.8.2008, počet zpráv: 3


27.8.2008 12:41:09 (#1)
gravatar

Pavel Machek

<pavel at suse.cz>
144
On Wed 2008-08-27 01:00:38, Petr Nejedly wrote: zobrazit citaci
> Pavel Machek napsal(a): > > Ahoj! > > > > je v priloze... > > > > Bohuzel tak jak je napsanej zvlada jen asi tak 2 adresy za sekundu > > :-(. grep ,19800, ho omezuje na jedno PSC, to asi dava smysl vyhodit > > nebo nahradit Vasim oblibenym PSC. > > Nespěchejmež...
Spechat se neda, pocitace jsou pomaly; ta konverze by mela trvat 10+ hodin... zobrazit citaci
> BH napsal(a): > > No, nejdriv bych to radsi prozkoumal, zjistil o kolik to nafoukne > > data, jak je to kvalitni, odlkadil to a pak teprve se rozhodoval > > Vzhledem k tomu, ze geotagovanych je zatim jen cca 10% adres, jedna se > o cca 300.000 nodu, nafouknuti CR o ~20%. Pokud se UIR_ADR pochlapi > a da to dohromady cele, narostla by nam CR o 200%. Pak bysme dosahli > paradoxniho stavu vicemene kompletni site silnic prvnich a druhych > trid a ulicni site, ale bez vetsiny silnic 3. tridy ;-)
Na pochlapeni UIR_ADR bych moc nespolehal. zobrazit citaci
> > jestli a jak se to tam nacpe. I kdyz si myslim ze ty data by tam byt v > > OSM mely. Dost to pomuze, jak pri mapovani, tak pri navigaci. > > Mely by tam byt urcite. Dulezite je doladit v jakem formatu a hlavne > nasetupovat proces pro updaty! (Precijen si nechceme zaneradit OSM > nejakymi 10%, ktere by nam pak vyrazneji komplikovali dodani tech > zbylych 90%...
Ten ADRESA_KOD by mel pro updaty stacit, ne? Anyway, tady je dalsi vzorek, mel by byt oznacen podle debaty na tady, takze pokud jsem neco udelal blbe, reknete... Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html ------------- další část --------------- cat > /tmp/delme.osm << EOF <?xml version='1.0' encoding='UTF-8'?> <osm version='0.5' generator='JOSM'> EOF ID=0 cat adresa.csv | grep -v ",,.$" | grep ,198.., | head -30 | ( while IFS=, read ADRESA_KOD OBJEKT_KOD ULICE_KOD CISOR_HOD CISOR_PIS PSC STAV VZNIK_DNE VZNIK_INFO ZANIK_DNE ZANIK_INFO PCD X Y; do ID=$[$ID-1] # echo $ULICE_KOD $CISOR_HOD $OBJEKT_KOD $PSC $X $Y if [ $STAV -eq 1 ]; then grep "^$ULICE_KOD" ulice.csv | ( IFS=, read I1 I2 NAME NAME2 grep "^$OBJEKT_KOD" objekt.csv | ( IFS=, read I1 I2 I3 CISDOM_HOD I4 echo "$X $Y" | ~/sf/timetab/cz/geo/jtsk2wgs | ( read LAT LON H echo $ID $NAME $CISDOM_HOD $CISOR_HOD $CISOR_PIS $PSC $LAT $LON echo "<node id='$ID' action='modify' visible='true' lat='$LAT' lon='$LON'>" >> /tmp/delme.osm echo " <tag k='addr:street' v=$NAME></tag>" >> /tmp/delme.osm echo " <tag k='addr:housenumber' v='$CISOR_HOD$CISOR_PIS'></tag>" >> /tmp/delme.osm echo " <tag k='addr:red' v='$CISDOM_HOD'></tag>" >> /tmp/delme.osm echo " <tag k='addr:postcode' v='$PSC'></tag>" >> /tmp/delme.osm echo " <tag k='uir_adr:ADRESA_KOD' v='$ADRESA_KOD'></tag>" >> /tmp/delme.osm echo " <tag k='source:addr' v='uir_adr'></tag>" >> /tmp/delme.osm echo "</node>" >> /tmp/delme.osm ) ) ) fi done ) cat >> /tmp/delme.osm << EOF </osm> EOF cat /tmp/delme.osm | cstocs 1250 utf8 > /tmp/delme2.osm ------------- další část --------------- <?xml version='1.0' encoding='UTF-8'?> <osm version='0.5' generator='JOSM'> <node id='-1' action='modify' visible='true' lat='50.1046381' lon='14.5605466'> <tag k='addr:street' v='Vlčkova'></tag> <tag k='addr:housenumber' v='2'></tag> <tag k='addr:red' v='1062'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22412883'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-2' action='modify' visible='true' lat='50.1047264' lon='14.5607255'> <tag k='addr:street' v='Vlčkova'></tag> <tag k='addr:housenumber' v='4'></tag> <tag k='addr:red' v='1063'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22412891'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-3' action='modify' visible='true' lat='50.1048502' lon='14.5608968'> <tag k='addr:street' v='Vlčkova'></tag> <tag k='addr:housenumber' v='6'></tag> <tag k='addr:red' v='1064'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22412905'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-4' action='modify' visible='true' lat='50.1049598' lon='14.5611135'> <tag k='addr:street' v='Vlčkova'></tag> <tag k='addr:housenumber' v='8'></tag> <tag k='addr:red' v='1065'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22412913'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-5' action='modify' visible='true' lat='50.1050557' lon='14.5612767'> <tag k='addr:street' v='Vlčkova'></tag> <tag k='addr:housenumber' v='10'></tag> <tag k='addr:red' v='1066'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22412921'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-6' action='modify' visible='true' lat='50.1054176' lon='14.5616800'> <tag k='addr:street' v='Vlčkova'></tag> <tag k='addr:housenumber' v='12'></tag> <tag k='addr:red' v='1067'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22412930'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-7' action='modify' visible='true' lat='50.1056088' lon='14.5615832'> <tag k='addr:street' v='Vlčkova'></tag> <tag k='addr:housenumber' v='3'></tag> <tag k='addr:red' v='1068'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22412948'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-8' action='modify' visible='true' lat='50.1054574' lon='14.5613048'> <tag k='addr:street' v='Vlčkova'></tag> <tag k='addr:housenumber' v='1'></tag> <tag k='addr:red' v='1069'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22412956'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-9' action='modify' visible='true' lat='50.1053301' lon='14.5607815'> <tag k='addr:street' v='Bratří VenclĂ­kĹŻ'></tag> <tag k='addr:housenumber' v='2'></tag> <tag k='addr:red' v='1070'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22412964'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-10' action='modify' visible='true' lat='50.1054861' lon='14.5605934'> <tag k='addr:street' v='Bratří VenclĂ­kĹŻ'></tag> <tag k='addr:housenumber' v='4'></tag> <tag k='addr:red' v='1071'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22412972'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-11' action='modify' visible='true' lat='50.1056445' lon='14.5604330'> <tag k='addr:street' v='Bratří VenclĂ­kĹŻ'></tag> <tag k='addr:housenumber' v='6'></tag> <tag k='addr:red' v='1072'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22412981'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-12' action='modify' visible='true' lat='50.1058118' lon='14.5602707'> <tag k='addr:street' v='Bratří VenclĂ­kĹŻ'></tag> <tag k='addr:housenumber' v='8'></tag> <tag k='addr:red' v='1073'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22412999'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-13' action='modify' visible='true' lat='50.1061687' lon='14.5597862'> <tag k='addr:street' v='Bratří VenclĂ­kĹŻ'></tag> <tag k='addr:housenumber' v='10'></tag> <tag k='addr:red' v='1074'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413006'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-14' action='modify' visible='true' lat='50.1062320' lon='14.5604077'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='2'></tag> <tag k='addr:red' v='1075'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413014'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-15' action='modify' visible='true' lat='50.1062680' lon='14.5606118'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='4'></tag> <tag k='addr:red' v='1076'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413022'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-16' action='modify' visible='true' lat='50.1063023' lon='14.5609008'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='6'></tag> <tag k='addr:red' v='1077'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413031'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-17' action='modify' visible='true' lat='50.1063621' lon='14.5611703'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='8'></tag> <tag k='addr:red' v='1078'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413049'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-18' action='modify' visible='true' lat='50.1065341' lon='14.5625167'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='10'></tag> <tag k='addr:red' v='1079'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413057'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-19' action='modify' visible='true' lat='50.1065599' lon='14.5627087'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='12'></tag> <tag k='addr:red' v='1080'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413065'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-20' action='modify' visible='true' lat='50.1066043' lon='14.5630097'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='14'></tag> <tag k='addr:red' v='1081'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413073'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-21' action='modify' visible='true' lat='50.1066641' lon='14.5632793'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='16'></tag> <tag k='addr:red' v='1082'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413081'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-22' action='modify' visible='true' lat='50.1068259' lon='14.5646136'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='18'></tag> <tag k='addr:red' v='1083'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413090'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-23' action='modify' visible='true' lat='50.1068542' lon='14.5648334'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='20'></tag> <tag k='addr:red' v='1084'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413103'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-24' action='modify' visible='true' lat='50.1069572' lon='14.5653901'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='24'></tag> <tag k='addr:red' v='1086'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413111'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-25' action='modify' visible='true' lat='50.1072822' lon='14.5674519'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='26'></tag> <tag k='addr:red' v='1087'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413120'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-26' action='modify' visible='true' lat='50.1073258' lon='14.5676402'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='28'></tag> <tag k='addr:red' v='1088'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413138'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-27' action='modify' visible='true' lat='50.1073869' lon='14.5679236'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='30'></tag> <tag k='addr:red' v='1089'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413146'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-28' action='modify' visible='true' lat='50.1074645' lon='14.5681894'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='32'></tag> <tag k='addr:red' v='1090'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413154'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-29' action='modify' visible='true' lat='50.1075521' lon='14.5687776'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='34'></tag> <tag k='addr:red' v='1091'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413162'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> <node id='-30' action='modify' visible='true' lat='50.1074264' lon='14.5690016'> <tag k='addr:street' v='CĂ­glerova'></tag> <tag k='addr:housenumber' v='36'></tag> <tag k='addr:red' v='1092'></tag> <tag k='addr:postcode' v='19800'></tag> <tag k='uir_adr:ADRESA_KOD' v='22413171'></tag> <tag k='source:addr' v='uir_adr'></tag> </node> </osm>

27.8.2008 01:09:41 (#2)
gravatar

Petr Nejedly

<Petr.Nejedly at Sun.COM>
111
Pavel Machek napsal(a): zobrazit citaci
> Spechat se neda, pocitace jsou pomaly; ta konverze by mela trvat 10+ hodin...
O to nejde. Jeste jsme se nedomluvili jak to ma vypadat a ty si tu hazis outer joinama nad CSV v bashi ;-) Stejne to nakonec nejlepe provede Tomas Kolda (vid ;-)) protoze uz ma v databazi i ty 3+ roky updatu a u nej ten outer join pobezi asi tak 130ms. zobrazit citaci
> Na pochlapeni UIR_ADR bych moc nespolehal.
Hmm, pravda, vsechny updaty dohromady daji necelych 24 tisic nove dodanych souradnic existujicich adres a vseho vsudy 9 (devet) novych adres ktere maji i souradnice. Takze z hlediska souradnic jsou relevantni jen updaty 442, 497, 606, 607 a 6 zobrazit citaci
> >> > jestli a jak se to tam nacpe. I kdyz si myslim ze ty data by tam byt v >> > OSM mely. Dost to pomuze, jak pri mapovani, tak pri navigaci. >> >> Mely by tam byt urcite. Dulezite je doladit v jakem formatu a hlavne >> nasetupovat proces pro updaty! (Precijen si nechceme zaneradit OSM >> nejakymi 10%, ktere by nam pak vyrazneji komplikovali dodani tech >> zbylych 90%... > > Ten ADRESA_KOD by mel pro updaty stacit, ne?
Ano zobrazit citaci
> Anyway, tady je dalsi vzorek, mel by byt oznacen podle debaty na > tady, takze pokud jsem neco udelal blbe, reknete...
Udelal. Vychazis ze 4 roky starych dat. Viz prvni odstavec. (Tim nechci nijak krotit tvoji kreativitu, jen ji mirne nasmerovat. Pokud to mergovani updatu taky napises v Bashi, jsi borec ;-) Teda ne ze by to neslo...) -- Petr "Nenik" Nejedly, NetBeans/Sun Microsystems, http://www.netbeans.org 355/113 -- Not the famous irrational number PI, but an incredible simulation!

27.8.2008 03:06:31 (#3)
gravatar

Pavel Machek

<pavel at ucw.cz>
1067 1226
On Wed 2008-08-27 13:09:41, Petr Nejedly wrote: zobrazit citaci
> Pavel Machek napsal(a): > > Spechat se neda, pocitace jsou pomaly; ta konverze by mela trvat 10+ hodin... > > O to nejde. Jeste jsme se nedomluvili jak to ma vypadat a ty si tu hazis > outer joinama nad CSV v bashi ;-)
;-). No, debata se zda-se omezila na 'jak se zapise cislo popisne', z cehoz usuzuju ze se v podstate shodujem.... zobrazit citaci
> Stejne to nakonec nejlepe provede Tomas Kolda (vid ;-)) protoze uz ma v databazi > i ty 3+ roky updatu a u nej ten outer join pobezi asi tak 130ms.
No, ono se to stejne bude uploadovat rychlosti 3 zaznamy za sekundu... ;-). No... kdyby nekde byly updatovany data v CSV, tak nepohrdnu; asi se mi update opravdu psat nechce. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

« zpět na výpis měsíce