egg-tcls: fetch_weather.pl annotate

annotate fetch_weather.pl @ 331:b8ae5e99341b

fetc_weather: Remove support for old HTML format parsing of Tiehallinto data, it's not provided in this format anymore.

author	Matti Hamalainen <ccr@tnsp.org>
date	Sat, 19 Sep 2015 16:09:12 +0300
parents	fe12434d6cbd
children	bff53f5bba5b

rev	line source
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	1 #!/usr/bin/perl -w
182 24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	2 ##########################################################################
24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	3 #
245 9435745b0dbb Bump version. Matti Hamalainen <ccr@tnsp.org> parents: 244 diff changeset	4 # Fetch Weather v0.3 by Matti 'ccr' Hamalainen <ccr@tnsp.org>
233 dc1afa011883 weather: Bump version, copyright. Matti Hamalainen <ccr@tnsp.org> parents: 232 diff changeset	5 # (C) Copyright 2014-2015 Tecnic Software productions (TNSP)
206 2ac661d551b3 Update documentation. Matti Hamalainen <ccr@tnsp.org> parents: 201 diff changeset	6 # This script is freely distributable under GNU GPL (version 2) license.
182 24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	7 #
24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	8 # Should be ran as a cronjob, and configured properly.
206 2ac661d551b3 Update documentation. Matti Hamalainen <ccr@tnsp.org> parents: 201 diff changeset	9 # /10 * * * perl -w /absolute/path/to/fetch_weather.pl /path/to/configfile
182 24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	10 #
186 b41c525bdaf7 Add example configuration file. Matti Hamalainen <ccr@tnsp.org> parents: 185 diff changeset	11 # Configuration file example is in fetch_weather.config
182 24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	12 #
206 2ac661d551b3 Update documentation. Matti Hamalainen <ccr@tnsp.org> parents: 201 diff changeset	13 # Requires various Perl modules, in Debian the packages should be:
2ac661d551b3 Update documentation. Matti Hamalainen <ccr@tnsp.org> parents: 201 diff changeset	14 # libwww-perl libxml-simple-perl libtimedate-perl
2ac661d551b3 Update documentation. Matti Hamalainen <ccr@tnsp.org> parents: 201 diff changeset	15 #
182 24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	16 #
24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	17 ##########################################################################
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	18 use strict;
243 aef2cf1dc3be Use UTF8 source. Matti Hamalainen <ccr@tnsp.org> parents: 242 diff changeset	19 use utf8;
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	20 use LWP::UserAgent;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	21 use HTML::Entities;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	22 use XML::Simple;
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	23 use Date::Format;
199 95012f8f687a Another round of timestamp parsing changes, urgh. Matti Hamalainen <ccr@tnsp.org> parents: 198 diff changeset	24 use Date::Parse;
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	25 use Data::Dumper;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	26
166 c8d5f3a7c4b7 Add debug option. Matti Hamalainen <ccr@tnsp.org> parents: 164 diff changeset	27
162 4f8a163b2bc1 Work on the backend weather data fetching. Matti Hamalainen <ccr@tnsp.org> parents: 158 diff changeset	28 ###
182 24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	29 ### Configuration settings
162 4f8a163b2bc1 Work on the backend weather data fetching. Matti Hamalainen <ccr@tnsp.org> parents: 158 diff changeset	30 ###
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	31 my %settings = (
166 c8d5f3a7c4b7 Add debug option. Matti Hamalainen <ccr@tnsp.org> parents: 164 diff changeset	32 "debug" => 0,
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	33 "opt_fmi" => 0,
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	34 "opt_tiehallinto" => 0,
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	35 "fmi_api_key" => "",
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	36 "outfile" => "",
185 64e06ee172c7 Add HTTP user agent setting. Matti Hamalainen <ccr@tnsp.org> parents: 182 diff changeset	37 "http_user_agent" => "Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 6.0) Opera 10.63 [en]",
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	38 );
162 4f8a163b2bc1 Work on the backend weather data fetching. Matti Hamalainen <ccr@tnsp.org> parents: 158 diff changeset	39
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	40
162 4f8a163b2bc1 Work on the backend weather data fetching. Matti Hamalainen <ccr@tnsp.org> parents: 158 diff changeset	41 ###
4f8a163b2bc1 Work on the backend weather data fetching. Matti Hamalainen <ccr@tnsp.org> parents: 158 diff changeset	42 ### Helper functions
4f8a163b2bc1 Work on the backend weather data fetching. Matti Hamalainen <ccr@tnsp.org> parents: 158 diff changeset	43 ###
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	44 sub mlog($)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	45 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	46 print STDERR $_[0];
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	47 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	48
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	49
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	50 sub fetch_http($)
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	51 {
198 60a4c69c7f25 Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 197 diff changeset	52 my $agent = LWP::UserAgent->new;
60a4c69c7f25 Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 197 diff changeset	53 $agent->agent($settings{"http_user_agent"});
208 c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	54 $agent->timeout(20);
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	55
198 60a4c69c7f25 Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 197 diff changeset	56 my $req = HTTP::Request->new(GET => $_[0]);
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	57
198 60a4c69c7f25 Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 197 diff changeset	58 return $agent->request($req);
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	59 }
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	60
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	61
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	62 sub parse_timestamp($$)
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	63 {
196 a38c123921a9 Change the timestamp parsing. Matti Hamalainen <ccr@tnsp.org> parents: 195 diff changeset	64 my ($str, $offs) = @_;
a38c123921a9 Change the timestamp parsing. Matti Hamalainen <ccr@tnsp.org> parents: 195 diff changeset	65 if ($str =~ /^(\d+):(\d+)$/)
a38c123921a9 Change the timestamp parsing. Matti Hamalainen <ccr@tnsp.org> parents: 195 diff changeset	66 {
a38c123921a9 Change the timestamp parsing. Matti Hamalainen <ccr@tnsp.org> parents: 195 diff changeset	67 return $offs + (60 * 60 * $1) + ($2 * 60);
a38c123921a9 Change the timestamp parsing. Matti Hamalainen <ccr@tnsp.org> parents: 195 diff changeset	68 }
a38c123921a9 Change the timestamp parsing. Matti Hamalainen <ccr@tnsp.org> parents: 195 diff changeset	69 else
a38c123921a9 Change the timestamp parsing. Matti Hamalainen <ccr@tnsp.org> parents: 195 diff changeset	70 {
a38c123921a9 Change the timestamp parsing. Matti Hamalainen <ccr@tnsp.org> parents: 195 diff changeset	71 return $offs;
a38c123921a9 Change the timestamp parsing. Matti Hamalainen <ccr@tnsp.org> parents: 195 diff changeset	72 }
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	73 }
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	74
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	75
194 55ed3db51ee7 Rename a function. Matti Hamalainen <ccr@tnsp.org> parents: 193 diff changeset	76 sub format_time_gmt($)
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	77 {
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	78 # 2012-02-27T00:00:00Z
199 95012f8f687a Another round of timestamp parsing changes, urgh. Matti Hamalainen <ccr@tnsp.org> parents: 198 diff changeset	79 return time2str("%Y-%m-%dT%TZ", $_[0], "UTC");
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	80 }
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	81
193 93ec73deebc5 Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 192 diff changeset	82
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	83 sub str_trim($)
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	84 {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	85 my $str = $_[0];
193 93ec73deebc5 Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 192 diff changeset	86 if (defined($str))
93ec73deebc5 Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 192 diff changeset	87 {
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	88 $str =~ s/^\s*//;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	89 $str =~ s/\s*$//;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	90 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	91 return $str;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	92 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	93
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	94
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	95 ###
331 b8ae5e99341b fetc_weather: Remove support for old HTML format parsing of Tiehallinto data, it's not provided in this format anymore. Matti Hamalainen <ccr@tnsp.org> parents: 325 diff changeset	96 ### Helper functions
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	97 ###
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	98
212 5f04ec60e8d9 weather: "Translate" rain state from Tiehallitus data. Matti Hamalainen <ccr@tnsp.org> parents: 208 diff changeset	99 my %th_rain_states =
5f04ec60e8d9 weather: "Translate" rain state from Tiehallitus data. Matti Hamalainen <ccr@tnsp.org> parents: 208 diff changeset	100 (
237 56af82fb7a6d Use lower case. Matti Hamalainen <ccr@tnsp.org> parents: 236 diff changeset	101 "Pouta" => "poutaa",
56af82fb7a6d Use lower case. Matti Hamalainen <ccr@tnsp.org> parents: 236 diff changeset	102 "Heikko" => "heikkoa sadetta",
56af82fb7a6d Use lower case. Matti Hamalainen <ccr@tnsp.org> parents: 236 diff changeset	103 "Kohtalainen" => "kohtalaista sadetta",
56af82fb7a6d Use lower case. Matti Hamalainen <ccr@tnsp.org> parents: 236 diff changeset	104 "Voimakas" => "voimakasta sadetta",
212 5f04ec60e8d9 weather: "Translate" rain state from Tiehallitus data. Matti Hamalainen <ccr@tnsp.org> parents: 208 diff changeset	105 );
5f04ec60e8d9 weather: "Translate" rain state from Tiehallitus data. Matti Hamalainen <ccr@tnsp.org> parents: 208 diff changeset	106
5f04ec60e8d9 weather: "Translate" rain state from Tiehallitus data. Matti Hamalainen <ccr@tnsp.org> parents: 208 diff changeset	107 my $th_rain_states_k = join("\|", map {quotemeta} sort { length($b)<=>length($a) } keys %th_rain_states);
5f04ec60e8d9 weather: "Translate" rain state from Tiehallitus data. Matti Hamalainen <ccr@tnsp.org> parents: 208 diff changeset	108
242 ffd49b583e34 Rename function. Matti Hamalainen <ccr@tnsp.org> parents: 238 diff changeset	109 sub translate_rain($)
212 5f04ec60e8d9 weather: "Translate" rain state from Tiehallitus data. Matti Hamalainen <ccr@tnsp.org> parents: 208 diff changeset	110 {
5f04ec60e8d9 weather: "Translate" rain state from Tiehallitus data. Matti Hamalainen <ccr@tnsp.org> parents: 208 diff changeset	111 my $tmp = $_[0];
237 56af82fb7a6d Use lower case. Matti Hamalainen <ccr@tnsp.org> parents: 236 diff changeset	112 $tmp =~ s/($th_rain_states_k)/$th_rain_states{$1}/igo;
212 5f04ec60e8d9 weather: "Translate" rain state from Tiehallitus data. Matti Hamalainen <ccr@tnsp.org> parents: 208 diff changeset	113 return $tmp;
5f04ec60e8d9 weather: "Translate" rain state from Tiehallitus data. Matti Hamalainen <ccr@tnsp.org> parents: 208 diff changeset	114 }
5f04ec60e8d9 weather: "Translate" rain state from Tiehallitus data. Matti Hamalainen <ccr@tnsp.org> parents: 208 diff changeset	115
331 b8ae5e99341b fetc_weather: Remove support for old HTML format parsing of Tiehallinto data, it's not provided in this format anymore. Matti Hamalainen <ccr@tnsp.org> parents: 325 diff changeset	116
244 7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	117 my %th_cloud_states =
7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	118 (
7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	119 0 => "selkeää",
247 85ad2912e720 Possibly handle cloudyness translation better. Matti Hamalainen <ccr@tnsp.org> parents: 245 diff changeset	120 1 => "melkein selkeää",
85ad2912e720 Possibly handle cloudyness translation better. Matti Hamalainen <ccr@tnsp.org> parents: 245 diff changeset	121 2 => "verrattain selkeää",
85ad2912e720 Possibly handle cloudyness translation better. Matti Hamalainen <ccr@tnsp.org> parents: 245 diff changeset	122 3 => "verrattain selkeää",
85ad2912e720 Possibly handle cloudyness translation better. Matti Hamalainen <ccr@tnsp.org> parents: 245 diff changeset	123 4 => "puolipilvistä",
85ad2912e720 Possibly handle cloudyness translation better. Matti Hamalainen <ccr@tnsp.org> parents: 245 diff changeset	124 5 => "verrattain pilvistä",
85ad2912e720 Possibly handle cloudyness translation better. Matti Hamalainen <ccr@tnsp.org> parents: 245 diff changeset	125 6 => "verrattain pilvistä",
244 7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	126 7 => "melkein pilvistä",
7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	127 8 => "pilvistä",
7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	128 );
7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	129
7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	130 sub translate_clouds($)
7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	131 {
247 85ad2912e720 Possibly handle cloudyness translation better. Matti Hamalainen <ccr@tnsp.org> parents: 245 diff changeset	132 return "" if ($_[0] eq "NaN" \|\| $_[0] eq "");
85ad2912e720 Possibly handle cloudyness translation better. Matti Hamalainen <ccr@tnsp.org> parents: 245 diff changeset	133 my $tmp = int($_[0]);
244 7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	134 foreach my $n (sort { $a <=> $b } keys %th_cloud_states)
7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	135 {
247 85ad2912e720 Possibly handle cloudyness translation better. Matti Hamalainen <ccr@tnsp.org> parents: 245 diff changeset	136 return $th_cloud_states{$n}." (".$n."/8)" if ($tmp == $n);
244 7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	137 }
7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	138 return $tmp;
7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	139 }
7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	140
212 5f04ec60e8d9 weather: "Translate" rain state from Tiehallitus data. Matti Hamalainen <ccr@tnsp.org> parents: 208 diff changeset	141
228 fe420576f284 The kludges intensify. Matti Hamalainen <ccr@tnsp.org> parents: 213 diff changeset	142 sub plonk_data($)
fe420576f284 The kludges intensify. Matti Hamalainen <ccr@tnsp.org> parents: 213 diff changeset	143 {
fe420576f284 The kludges intensify. Matti Hamalainen <ccr@tnsp.org> parents: 213 diff changeset	144 return defined($_[0]) ? $_[0] : "";
fe420576f284 The kludges intensify. Matti Hamalainen <ccr@tnsp.org> parents: 213 diff changeset	145 }
fe420576f284 The kludges intensify. Matti Hamalainen <ccr@tnsp.org> parents: 213 diff changeset	146
fe420576f284 The kludges intensify. Matti Hamalainen <ccr@tnsp.org> parents: 213 diff changeset	147
237 56af82fb7a6d Use lower case. Matti Hamalainen <ccr@tnsp.org> parents: 236 diff changeset	148 sub plonk_data_lc($)
56af82fb7a6d Use lower case. Matti Hamalainen <ccr@tnsp.org> parents: 236 diff changeset	149 {
56af82fb7a6d Use lower case. Matti Hamalainen <ccr@tnsp.org> parents: 236 diff changeset	150 return defined($_[0]) ? lc($_[0]) : "";
56af82fb7a6d Use lower case. Matti Hamalainen <ccr@tnsp.org> parents: 236 diff changeset	151 }
56af82fb7a6d Use lower case. Matti Hamalainen <ccr@tnsp.org> parents: 236 diff changeset	152
56af82fb7a6d Use lower case. Matti Hamalainen <ccr@tnsp.org> parents: 236 diff changeset	153
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	154 ###
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	155 ### Configuration handling
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	156 ###
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	157 sub opt_chk_bool($)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	158 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	159 if (defined($settings{$_[0]}))
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	160 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	161 my $val = $settings{$_[0]};
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	162 return ($val == 1 \|\| $val eq "true" \|\| $val eq "on" \|\| $val eq "1");
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	163 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	164 else
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	165 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	166 return 0;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	167 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	168 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	169
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	170
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	171 sub opt_chk_valid($$)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	172 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	173 if (defined($settings{$_[0]}))
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	174 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	175 my $val = $settings{$_[0]};
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	176 return length($val) >= $_[1];
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	177 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	178 else
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	179 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	180 return 0;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	181 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	182 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	183
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	184
324 19c64798de91 weather: Add some better optional debugging. Matti Hamalainen <ccr@tnsp.org> parents: 247 diff changeset	185 sub opt_get_int($)
19c64798de91 weather: Add some better optional debugging. Matti Hamalainen <ccr@tnsp.org> parents: 247 diff changeset	186 {
19c64798de91 weather: Add some better optional debugging. Matti Hamalainen <ccr@tnsp.org> parents: 247 diff changeset	187 if (defined($settings{$_[0]}))
19c64798de91 weather: Add some better optional debugging. Matti Hamalainen <ccr@tnsp.org> parents: 247 diff changeset	188 {
19c64798de91 weather: Add some better optional debugging. Matti Hamalainen <ccr@tnsp.org> parents: 247 diff changeset	189 return int($settings{$_[0]});
19c64798de91 weather: Add some better optional debugging. Matti Hamalainen <ccr@tnsp.org> parents: 247 diff changeset	190 }
19c64798de91 weather: Add some better optional debugging. Matti Hamalainen <ccr@tnsp.org> parents: 247 diff changeset	191 else
19c64798de91 weather: Add some better optional debugging. Matti Hamalainen <ccr@tnsp.org> parents: 247 diff changeset	192 {
19c64798de91 weather: Add some better optional debugging. Matti Hamalainen <ccr@tnsp.org> parents: 247 diff changeset	193 return -1;
19c64798de91 weather: Add some better optional debugging. Matti Hamalainen <ccr@tnsp.org> parents: 247 diff changeset	194 }
19c64798de91 weather: Add some better optional debugging. Matti Hamalainen <ccr@tnsp.org> parents: 247 diff changeset	195 }
19c64798de91 weather: Add some better optional debugging. Matti Hamalainen <ccr@tnsp.org> parents: 247 diff changeset	196
19c64798de91 weather: Add some better optional debugging. Matti Hamalainen <ccr@tnsp.org> parents: 247 diff changeset	197
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	198 sub opt_get($)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	199 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	200 if (defined($settings{$_[0]}))
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	201 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	202 return $settings{$_[0]};
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	203 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	204 else
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	205 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	206 return undef;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	207 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	208 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	209
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	210
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	211 sub opt_read_config($)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	212 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	213 my $filename = $_[0];
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	214 my $errors = 0;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	215 my $line = 0;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	216
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	217 open(CONFFILE, "<", $filename) or die("Could not open configuration '".$filename."'!\n");
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	218 while (<CONFFILE>)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	219 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	220 $line++;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	221 chomp;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	222 if (/(^\s#\|^\s$)/) {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	223 # Ignore comments and empty lines
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	224 } elsif (/^\s\"?([a-zA-Z0-9_]+)\"?\s=>?\s(\d+),?\s$/) {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	225 my $key = lc($1);
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	226 my $value = $2;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	227 if (defined($settings{$key})) {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	228 $settings{$key} = $value;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	229 } else {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	230 mlog("[$filename:$line] Unknown setting '$key' = $value\n");
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	231 $errors = 1;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	232 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	233 } elsif (/^\s\"?([a-zA-Z0-9_]+)\"?\s=>?\s\"(.?)\",?\s*$/) {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	234 my $key = lc($1);
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	235 my $value = $2;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	236 if (defined($settings{$key})) {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	237 $settings{$key} = $value;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	238 } else {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	239 mlog("[$filename:$line] Unknown setting '$key' = '$value'\n");
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	240 $errors = 1;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	241 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	242 } else {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	243 mlog("[$filename:$line] Syntax error: $_\n");
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	244 $errors = 1;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	245 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	246 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	247 close(CONFFILE);
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	248 return $errors;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	249 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	250
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	251
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	252 ###
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	253 ### Main program begins
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	254 ###
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	255 my $weatherdata = {};
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	256
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	257 die(
325 fe12434d6cbd weather: Bump version. Matti Hamalainen <ccr@tnsp.org> parents: 324 diff changeset	258 "Weather Fetch v0.3 by ccr/TNSP <ccr\@tnsp.org>\n".
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	259 "Usage: $0 <config file>\n"
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	260 ) unless scalar(@ARGV) >= 1;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	261
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	262 my $cfgfile = shift;
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	263 opt_read_config($cfgfile) == 0 or die("Errors while parsing configuration file '".$cfgfile."'.\n");
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	264
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	265
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	266 ###
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	267 ### Fetch FMI data
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	268 ###
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	269 if (opt_chk_bool("opt_fmi"))
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	270 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	271 die("FMI data scrape enabled, but no API key set.\n") unless opt_chk_valid("fmi_api_key", 10);
238 a1d6e2d8789e Add new data from FMI service and change datafile format. Matti Hamalainen <ccr@tnsp.org> parents: 237 diff changeset	272 my @fmitems = ("temperature", "humidity", "windspeedms", "totalcloudcover");
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	273
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	274 my $uri = "http://data.fmi.fi/fmi-apikey/".opt_get("fmi_api_key").
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	275 "/wfs?request=getFeature&storedquery_id=fmi::observations::weather::".
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	276 "multipointcoverage".
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	277 # "timevaluepair".
194 55ed3db51ee7 Rename a function. Matti Hamalainen <ccr@tnsp.org> parents: 193 diff changeset	278 "&starttime=".format_time_gmt(time() - 10*60)."&endtime=".format_time_gmt(time()).
238 a1d6e2d8789e Add new data from FMI service and change datafile format. Matti Hamalainen <ccr@tnsp.org> parents: 237 diff changeset	279 "&parameters=".join(",", @fmitems)."&maxlocations=100&bbox=19,59,32,75";
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	280
324 19c64798de91 weather: Add some better optional debugging. Matti Hamalainen <ccr@tnsp.org> parents: 247 diff changeset	281 print STDERR "FMI URI: ".$uri."\n" if (opt_get_int("debug") > 0);
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	282
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	283 my $res = fetch_http($uri);
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	284 if ($res->code >= 200 && $res->code <= 201)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	285 {
170 e4faa3746869 Remove useless parameters from XMLin(). Matti Hamalainen <ccr@tnsp.org> parents: 169 diff changeset	286 my $xml = XMLin($res->decoded_content);
169 c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	287 my $time_base = time();
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	288
208 c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	289 if (defined($xml->{"wfs:member"}{"omso:GridSeriesObservation"}))
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	290 {
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	291 my $fdata = $xml->{"wfs:member"}{"omso:GridSeriesObservation"};
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	292 my $fshit = $fdata->{"om:result"}{"gmlcov:MultiPointCoverage"};
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	293 my @farray = ();
169 c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	294
208 c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	295 foreach my $fline (split(/\n/, $fshit->{"gml:domainSet"}{"gmlcov:SimpleMultiPoint"}{"gmlcov:positions"}))
169 c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	296 {
208 c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	297 if ($fline =~ /^\s([\+\-]?\d+\.\d)\s+([\+\-]?\d+\.\d)\s+(\d+)\s$/)
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	298 {
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	299 push(@farray, {"lat" => $1, "long" => $2, "time" => $3});
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	300 }
169 c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	301 }
c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	302
208 c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	303 my $findex = 0;
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	304 foreach my $fline (split(/\n/, $fshit->{"gml:rangeSet"}{"gml:DataBlock"}{"gml:doubleOrNilReasonTupleList"}))
169 c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	305 {
238 a1d6e2d8789e Add new data from FMI service and change datafile format. Matti Hamalainen <ccr@tnsp.org> parents: 237 diff changeset	306 my @fmatches = ($fline =~ /\s([\+\-]?\d+\.\d\|NaN)\s*/ig);
a1d6e2d8789e Add new data from FMI service and change datafile format. Matti Hamalainen <ccr@tnsp.org> parents: 237 diff changeset	307 if (scalar(@fmatches) > 0)
208 c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	308 {
238 a1d6e2d8789e Add new data from FMI service and change datafile format. Matti Hamalainen <ccr@tnsp.org> parents: 237 diff changeset	309 die("Not enough items in scalar line (".scalar(@fmatches). " vs ".scalar(@fmitems).
a1d6e2d8789e Add new data from FMI service and change datafile format. Matti Hamalainen <ccr@tnsp.org> parents: 237 diff changeset	310 "): ".$fline."\n") if (scalar(@fmatches) != scalar(@fmitems));
a1d6e2d8789e Add new data from FMI service and change datafile format. Matti Hamalainen <ccr@tnsp.org> parents: 237 diff changeset	311 for (my $fni = 0; $fni < scalar(@fmitems); $fni++)
a1d6e2d8789e Add new data from FMI service and change datafile format. Matti Hamalainen <ccr@tnsp.org> parents: 237 diff changeset	312 {
a1d6e2d8789e Add new data from FMI service and change datafile format. Matti Hamalainen <ccr@tnsp.org> parents: 237 diff changeset	313 $farray[$findex]{$fmitems[$fni]} = $fmatches[$fni] if (lc($fmatches[$fni]) ne "nan");
a1d6e2d8789e Add new data from FMI service and change datafile format. Matti Hamalainen <ccr@tnsp.org> parents: 237 diff changeset	314 }
208 c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	315 $findex++;
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	316 }
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	317 }
208 c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	318 # XXX Hashify the array into lat/long keys
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	319
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	320 # This is horrible :S
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	321 my $fcrap = $fdata->{"om:featureOfInterest"}{"sams:SF_SpatialSamplingFeature"}{"sams:shape"}{"gml:MultiPoint"}{"gml:pointMember"};
228 fe420576f284 The kludges intensify. Matti Hamalainen <ccr@tnsp.org> parents: 213 diff changeset	322 foreach my $xnode (@{$fcrap})
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	323 {
228 fe420576f284 The kludges intensify. Matti Hamalainen <ccr@tnsp.org> parents: 213 diff changeset	324 my $floc = $xnode->{"gml:Point"};
208 c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	325 if ($floc->{"gml:pos"} =~ /^\s([\+\-]?\d+\.\d)\s+([\+\-]?\d+\.\d)\s$/)
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	326 {
208 c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	327 my ($flat, $flong) = ($1, $2);
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	328 # Should use a hash -
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	329 foreach my $flol (@farray)
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	330 {
208 c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	331 if ($flol->{"lat"} == $flat && $flol->{"long"} == $flong)
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	332 {
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	333 $weatherdata->{$floc->{"gml:name"}} =
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	334 [
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	335 1,
228 fe420576f284 The kludges intensify. Matti Hamalainen <ccr@tnsp.org> parents: 213 diff changeset	336 plonk_data($flol->{"time"}),
237 56af82fb7a6d Use lower case. Matti Hamalainen <ccr@tnsp.org> parents: 236 diff changeset	337 plonk_data($flol->{"temperature"}),
238 a1d6e2d8789e Add new data from FMI service and change datafile format. Matti Hamalainen <ccr@tnsp.org> parents: 237 diff changeset	338
228 fe420576f284 The kludges intensify. Matti Hamalainen <ccr@tnsp.org> parents: 213 diff changeset	339 plonk_data($flol->{"humidity"}),
237 56af82fb7a6d Use lower case. Matti Hamalainen <ccr@tnsp.org> parents: 236 diff changeset	340 plonk_data($flol->{"windspeedms"}),
244 7dea85b9a431 Translate cloudiness data. Matti Hamalainen <ccr@tnsp.org> parents: 243 diff changeset	341 translate_clouds(plonk_data($flol->{"totalcloudcover"})),
208 c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	342 ];
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	343 }
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	344 }
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	345 }
169 c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	346 }
208 c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	347 } else {
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	348 # defined
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	349 print STDERR "Invalid XML received:\n";
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	350 print STDERR $res->decoded_content."\n\n";
169 c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	351 }
208 c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	352 } else {
c6bbf69afc57 Add some error handling and debugging. Matti Hamalainen <ccr@tnsp.org> parents: 206 diff changeset	353 print STDERR "Error fetching FMI XML: ".$res->status_line."\n";
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	354 }
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	355 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	356
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	357
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	358 ###
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	359 ### Output
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	360 ###
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	361 if (opt_chk_valid("outfile", 1)) {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	362 open(STDOUT, '>', opt_get("outfile")) or die("Could not open output file '".opt_get("outfile")."'.\n");
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	363 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	364
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	365 binmode STDOUT, ':encoding(utf-8)';
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	366
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	367 foreach my $key (sort { $a cmp $b } keys %$weatherdata)
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	368 {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	369 print STDOUT $key."\|".join("\|", @{$weatherdata->{$key}})."\n";
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	370 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	371
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	372 close(STDOUT);

Mercurial > hg > egg-tcls

annotate fetch_weather.pl @ 331:b8ae5e99341b