egg-tcls: fetch_weather.pl annotate

annotate fetch_weather.pl @ 195:9eee4e1b757c

Slight adjustments.

author	Matti Hamalainen <ccr@tnsp.org>
date	Mon, 16 Jun 2014 01:54:56 +0300
parents	55ed3db51ee7
children	a38c123921a9

rev	line source
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	1 #!/usr/bin/perl -w
182 24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	2 ##########################################################################
24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	3 #
24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	4 # Fetch Weather v0.1 by Matti 'ccr' Hamalainen <ccr@tnsp.org>
24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	5 # (C) Copyright 2014 Tecnic Software productions (TNSP)
24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	6 #
24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	7 # Should be ran as a cronjob, and configured properly.
24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	8 # 15 * * * * /absolute/path/to/fetch_weather.pl </path/to/configfile>
24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	9 #
186 b41c525bdaf7 Add example configuration file. Matti Hamalainen <ccr@tnsp.org> parents: 185 diff changeset	10 # Configuration file example is in fetch_weather.config
182 24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	11 #
24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	12 # This script is freely distributable under GNU GPL (version 2) license.
24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	13 #
24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	14 ##########################################################################
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	15 use strict;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	16 use LWP::UserAgent;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	17 use HTML::Entities;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	18 use XML::Simple;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	19 use Text::Iconv;
191 127b1d0f874e Use Time::Piece for some timestamp parsing. Matti Hamalainen <ccr@tnsp.org> parents: 190 diff changeset	20 use Time::Piece;
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	21 use Date::Format;
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	22 use Data::Dumper;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	23
166 c8d5f3a7c4b7 Add debug option. Matti Hamalainen <ccr@tnsp.org> parents: 164 diff changeset	24
162 4f8a163b2bc1 Work on the backend weather data fetching. Matti Hamalainen <ccr@tnsp.org> parents: 158 diff changeset	25 ###
182 24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	26 ### Configuration settings
162 4f8a163b2bc1 Work on the backend weather data fetching. Matti Hamalainen <ccr@tnsp.org> parents: 158 diff changeset	27 ###
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	28 my %settings = (
166 c8d5f3a7c4b7 Add debug option. Matti Hamalainen <ccr@tnsp.org> parents: 164 diff changeset	29 "debug" => 0,
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	30 "opt_fmi" => 0,
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	31 "opt_tiehallinto" => 0,
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	32 "fmi_api_key" => "",
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	33 "outfile" => "",
185 64e06ee172c7 Add HTTP user agent setting. Matti Hamalainen <ccr@tnsp.org> parents: 182 diff changeset	34 "http_user_agent" => "Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 6.0) Opera 10.63 [en]",
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	35 );
162 4f8a163b2bc1 Work on the backend weather data fetching. Matti Hamalainen <ccr@tnsp.org> parents: 158 diff changeset	36
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	37
162 4f8a163b2bc1 Work on the backend weather data fetching. Matti Hamalainen <ccr@tnsp.org> parents: 158 diff changeset	38 ###
4f8a163b2bc1 Work on the backend weather data fetching. Matti Hamalainen <ccr@tnsp.org> parents: 158 diff changeset	39 ### Helper functions
4f8a163b2bc1 Work on the backend weather data fetching. Matti Hamalainen <ccr@tnsp.org> parents: 158 diff changeset	40 ###
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	41 sub mlog($)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	42 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	43 print STDERR $_[0];
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	44 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	45
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	46
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	47 sub fetch_http($)
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	48 {
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	49 my $tmpAgent = LWP::UserAgent->new;
185 64e06ee172c7 Add HTTP user agent setting. Matti Hamalainen <ccr@tnsp.org> parents: 182 diff changeset	50 $tmpAgent->agent($settings{"http_user_agent"});
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	51 $tmpAgent->timeout(10);
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	52
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	53 my $tmpRequest = HTTP::Request->new(GET => $_[0]);
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	54
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	55 return $tmpAgent->request($tmpRequest);
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	56 }
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	57
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	58
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	59 sub parse_timestamp($$)
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	60 {
182 24f9f38210fe Add comments. Matti Hamalainen <ccr@tnsp.org> parents: 178 diff changeset	61 # XXX: A bit of a hack, but it works.
195 9eee4e1b757c Slight adjustments. Matti Hamalainen <ccr@tnsp.org> parents: 194 diff changeset	62 return Time::Piece->strptime($_[0], "%H:%M")->epoch + $_[1];
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	63 }
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	64
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	65
194 55ed3db51ee7 Rename a function. Matti Hamalainen <ccr@tnsp.org> parents: 193 diff changeset	66 sub format_time_gmt($)
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	67 {
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	68 # 2012-02-27T00:00:00Z
191 127b1d0f874e Use Time::Piece for some timestamp parsing. Matti Hamalainen <ccr@tnsp.org> parents: 190 diff changeset	69 return gmtime($_[0])->strftime("%Y-%m-%dT%TZ");
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	70 }
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	71
193 93ec73deebc5 Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 192 diff changeset	72
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	73 sub str_trim($)
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	74 {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	75 my $str = $_[0];
193 93ec73deebc5 Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 192 diff changeset	76 if (defined($str))
93ec73deebc5 Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 192 diff changeset	77 {
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	78 $str =~ s/^\s*//;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	79 $str =~ s/\s*$//;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	80 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	81 return $str;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	82 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	83
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	84
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	85 ###
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	86 ### Loose HTML parser
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	87 ###
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	88 sub pop_token_a($)
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	89 {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	90 my $tokens = $_[0];
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	91 return shift(@$tokens);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	92 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	93
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	94
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	95 sub pop_token($)
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	96 {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	97 return str_trim(pop_token_a($_[0]));
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	98 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	99
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	100
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	101 sub parse_html_str($)
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	102 {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	103 my $tokens = $_[0];
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	104 my $token = pop_token($tokens);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	105 my $str = "";
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	106 $token =~ s/^\s*//;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	107
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	108 return undef unless (substr($token, 0, 1) eq '"');
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	109 $token = substr($token, 1);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	110
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	111 while (defined($token)) {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	112 my $tmp = $token;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	113 $tmp =~ s/\s*$//;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	114 if (substr($tmp, -1) eq '"') {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	115 $str .= substr($tmp, 0, -1);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	116 return $str;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	117 } else {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	118 $str .= $token;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	119 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	120 $token = shift(@$tokens);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	121 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	122 return undef;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	123 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	124
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	125
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	126 sub parse_html_tree($$);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	127
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	128 sub parse_html_tree($$)
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	129 {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	130 my ($tokens, $tree) = @_;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	131
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	132 while (my $token = pop_token($tokens)) {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	133 if ($token =~ /^<[!\/]?[a-zA-Z]+/) {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	134 $token = lc($token);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	135 if ($token =~ /^<\!.*>$/) {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	136 # Ignore comments etc.
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	137 } elsif ($token =~ /^<([a-z]+[1-9])(.)>$/) {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	138 my ($name, $args) = ($1, $2);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	139 if ($name eq "meta" \|\| $name eq "img") {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	140 my $tmp = {};
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	141 parse_html_tree($tokens, $tree);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	142 return $tree;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	143 } else {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	144 my $tmp = { "name" => $name, "args" => str_trim($args) };
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	145 parse_html_tree($tokens, $tmp);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	146 push(@{$$tree{"nodes"}}, $tmp);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	147 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	148 } elsif ($token =~ /^<\/([a-z]+[1-9]*)>$/) {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	149 return $tree;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	150 } else {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	151 die("HORROR TERROR ELITE: '$token'\n");
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	152 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	153 } else {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	154 $token = str_trim(decode_entities($token));
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	155 push(@{$$tree{"nodes"}}, { "name" => "text", "args" => "", "text" => $token }) if length($token) > 0;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	156 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	157 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	158
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	159 return $tree;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	160 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	161
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	162
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	163 sub parse_html($)
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	164 {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	165 return undef unless defined($_[0]);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	166 my $str = $_[0];
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	167 my $res = { "name" => "", "args" => "" };
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	168 $str =~ tr/\r/ /;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	169 $str =~ tr/\n/ /;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	170 my @tokens = grep { !($_ =~ /^\s$/) } split(/(<\/?[a-zA-Z]+.?>)/, $str);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	171 parse_html_tree(\@tokens, $res);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	172 return $res;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	173 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	174
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	175 sub html_find_node($$$);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	176
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	177 sub html_find_node($$$)
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	178 {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	179 my ($node, $name, $args) = @_;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	180
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	181 if (defined($node)) {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	182 if (ref($node) eq "ARRAY") {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	183 foreach my $n (@$node) {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	184 my $tmp = html_find_node($n, $name, $args);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	185 # Must do it like this, in order not to break the loop
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	186 return $tmp if defined($tmp);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	187 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	188 } elsif (ref($node) eq "HASH") {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	189 if (defined($$node{"name"})) {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	190 if ($$node{"name"} eq $name) {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	191 if ($args ne "") {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	192 if (defined($$node{"args"}) && $$node{"args"} =~ /$args/) {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	193 } else {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	194 return html_find_node($$node{"nodes"}, $name, $args);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	195 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	196 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	197 return $node;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	198 } else {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	199 return html_find_node($$node{"nodes"}, $name, $args);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	200 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	201 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	202 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	203 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	204 return undef;
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	205 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	206
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	207
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	208 ###
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	209 ### Helper functions for locating/traversing nodes
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	210 ### in the parsed data tree structure.
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	211 ###
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	212 sub fnode($$)
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	213 {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	214 return html_find_node($_[0], $_[1], "");
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	215 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	216
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	217
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	218 sub fnodea($$$)
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	219 {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	220 return html_find_node($_[0], $_[1], $_[2]);
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	221 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	222
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	223
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	224 sub get_node($$$)
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	225 {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	226 return defined($_[0]->[$_[2]]{"nodes"}[0]{$_[1]}) ? $_[0]->[$_[2]]{"nodes"}[0]{$_[1]} : "";
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	227 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	228
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	229
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	230 ###
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	231 ### Configuration handling
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	232 ###
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	233 sub opt_chk_bool($)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	234 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	235 if (defined($settings{$_[0]}))
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	236 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	237 my $val = $settings{$_[0]};
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	238 return ($val == 1 \|\| $val eq "true" \|\| $val eq "on" \|\| $val eq "1");
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	239 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	240 else
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	241 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	242 return 0;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	243 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	244 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	245
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	246
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	247 sub opt_chk_valid($$)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	248 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	249 if (defined($settings{$_[0]}))
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	250 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	251 my $val = $settings{$_[0]};
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	252 return length($val) >= $_[1];
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	253 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	254 else
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	255 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	256 return 0;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	257 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	258 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	259
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	260
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	261 sub opt_get($)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	262 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	263 if (defined($settings{$_[0]}))
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	264 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	265 return $settings{$_[0]};
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	266 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	267 else
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	268 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	269 return undef;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	270 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	271 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	272
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	273
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	274 sub opt_read_config($)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	275 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	276 my $filename = $_[0];
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	277 my $errors = 0;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	278 my $line = 0;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	279
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	280 open(CONFFILE, "<", $filename) or die("Could not open configuration '".$filename."'!\n");
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	281 while (<CONFFILE>)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	282 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	283 $line++;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	284 chomp;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	285 if (/(^\s#\|^\s$)/) {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	286 # Ignore comments and empty lines
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	287 } elsif (/^\s\"?([a-zA-Z0-9_]+)\"?\s=>?\s(\d+),?\s$/) {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	288 my $key = lc($1);
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	289 my $value = $2;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	290 if (defined($settings{$key})) {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	291 $settings{$key} = $value;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	292 } else {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	293 mlog("[$filename:$line] Unknown setting '$key' = $value\n");
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	294 $errors = 1;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	295 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	296 } elsif (/^\s\"?([a-zA-Z0-9_]+)\"?\s=>?\s\"(.?)\",?\s*$/) {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	297 my $key = lc($1);
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	298 my $value = $2;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	299 if (defined($settings{$key})) {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	300 $settings{$key} = $value;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	301 } else {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	302 mlog("[$filename:$line] Unknown setting '$key' = '$value'\n");
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	303 $errors = 1;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	304 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	305 } else {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	306 mlog("[$filename:$line] Syntax error: $_\n");
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	307 $errors = 1;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	308 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	309 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	310 close(CONFFILE);
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	311 return $errors;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	312 }
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	313
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	314
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	315 ###
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	316 ### Main program begins
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	317 ###
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	318 my $weatherdata = {};
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	319
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	320 die(
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	321 "Weather Fetch v0.1 by ccr/TNSP <ccr\@tnsp.org>\n".
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	322 "Usage: $0 <config file>\n"
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	323 ) unless scalar(@ARGV) >= 1;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	324
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	325 my $cfgfile = shift;
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	326 opt_read_config($cfgfile) == 0 or die("Errors while parsing configuration file '".$cfgfile."'.\n");
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	327
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	328
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	329 ###
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	330 ### Fetch tiehallinto road weather measurement data
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	331 ###
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	332 if (opt_chk_bool("opt_tiehallinto"))
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	333 {
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	334 for (my $i = 1; $i <= 22; $i++)
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	335 {
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	336 my $res = fetch_http("http://alk.tiehallinto.fi/alk/tiesaa/tiesaa_maak_".$i.".html");
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	337 if ($res->code >= 200 && $res->code <= 201)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	338 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	339 my $data = $res->decoded_content;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	340
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	341 # Filter out crap tags we don't want or need
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	342 $data =~ s/\n/§/g;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	343 $data =~ s/<!--.*?-->//ig;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	344 $data =~ s/<map[^>]>.?<\/map>//ig;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	345 $data =~ s/<form[^>]>.?<\/form>//ig;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	346 $data =~ s/<script[^>]>.?<\/script>//ig;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	347 $data =~ s/<meta[^>]*>//ig;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	348 $data =~ s/<font[^>]*>//ig;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	349 $data =~ s/<\/font>//ig;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	350 $data =~ s/<span[^>]*>//ig;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	351 $data =~ s/<\/span>//ig;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	352 $data =~ s/<\/?b>//ig;
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	353
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	354 $data =~ s/<br>//ig;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	355 $data =~ s/ / /ig;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	356 $data =~ s/§/\n/g;
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	357
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	358 # Parse the HTML mess
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	359 my $otree = parse_html($data);
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	360
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	361 # Find our desired element nodes
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	362 my $odata = fnodea(fnode($otree, "body"), "div", "class=elementc");
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	363 my $oupdate = fnode($odata, "p");
191 127b1d0f874e Use Time::Piece for some timestamp parsing. Matti Hamalainen <ccr@tnsp.org> parents: 190 diff changeset	364 my $time_base = Time::Piece->strptime("00:00", "%H:%M")->epoch;
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	365 if ($oupdate) {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	366 my $tmp = $oupdate->{"nodes"}[0]{"text"};
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	367 if ($tmp =~ /:\s+(\d\d\.\d\d\.\d\d\d\d)\s+(\d\d:\d\d)/) {
195 9eee4e1b757c Slight adjustments. Matti Hamalainen <ccr@tnsp.org> parents: 194 diff changeset	368 $time_base = Time::Piece->strptime($1." 00:00", "%d.%m.%Y %H:%M")->epoch;
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	369 }
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	370 }
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	371
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	372 my $oelems = fnode($odata, "table")->{"nodes"};
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	373 for (my $n = 1; $n < scalar(@$oelems); $n++)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	374 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	375 my $fdata = @$oelems[$n]->{"nodes"};
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	376 $weatherdata->{$fdata->[0]{"nodes"}[0]{"text"}} =
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	377 [
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	378 parse_timestamp(get_node($fdata, "text", 1), $time_base),
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	379 get_node($fdata, "text", 2),
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	380 get_node($fdata, "text", 3),
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	381 get_node($fdata, "text", 4),
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	382 ];
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	383 }
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	384 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	385 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	386 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	387
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	388
9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	389 ###
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	390 ### Fetch FMI data
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	391 ###
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	392 if (opt_chk_bool("opt_fmi"))
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	393 {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	394 die("FMI data scrape enabled, but no API key set.\n") unless opt_chk_valid("fmi_api_key", 10);
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	395
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	396 my $uri = "http://data.fmi.fi/fmi-apikey/".opt_get("fmi_api_key").
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	397 "/wfs?request=getFeature&storedquery_id=fmi::observations::weather::".
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	398 "multipointcoverage".
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	399 # "timevaluepair".
194 55ed3db51ee7 Rename a function. Matti Hamalainen <ccr@tnsp.org> parents: 193 diff changeset	400 "&starttime=".format_time_gmt(time() - 10*60)."&endtime=".format_time_gmt(time()).
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	401 "&parameters=temperature,humidity&maxlocations=50&bbox=19,59,32,70";
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	402
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	403 print STDERR "FMI URI: ".$uri."\n" if opt_chk_bool("debug");
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	404
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	405 my $res = fetch_http($uri);
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	406 if ($res->code >= 200 && $res->code <= 201)
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	407 {
170 e4faa3746869 Remove useless parameters from XMLin(). Matti Hamalainen <ccr@tnsp.org> parents: 169 diff changeset	408 my $xml = XMLin($res->decoded_content);
169 c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	409 my $time_base = time();
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	410 my $fdata = $xml->{"wfs:member"}{"omso:GridSeriesObservation"};
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	411
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	412 my $fshit = $fdata->{"om:result"}{"gmlcov:MultiPointCoverage"};
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	413 my @farray = ();
169 c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	414
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	415 foreach my $fline (split(/\n/, $fshit->{"gml:domainSet"}{"gmlcov:SimpleMultiPoint"}{"gmlcov:positions"}))
169 c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	416 {
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	417 if ($fline =~ /^\s([\+\-]?\d+\.\d)\s+([\+\-]?\d+\.\d)\s+(\d+)\s$/)
169 c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	418 {
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	419 push(@farray, {"lat" => $1, "long" => $2, "time" => $3});
169 c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	420 }
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	421 }
169 c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	422
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	423 my $findex = 0;
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	424 foreach my $fline (split(/\n/, $fshit->{"gml:rangeSet"}{"gml:DataBlock"}{"gml:doubleOrNilReasonTupleList"}))
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	425 {
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	426 if ($fline =~ /^\s([\+\-]?\d+\.\d\|NaN)\s+([\+\-]?\d+\.\d\|NaN)\s$/)
169 c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	427 {
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	428 $farray[$findex]{"temp"} = $1 if ($1 ne "NaN");
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	429 $farray[$findex]{"humidity"} = $2 if ($2 ne "NaN");
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	430 $findex++;
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	431 }
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	432 }
190 15c49bb1ce05 Add some comments. Matti Hamalainen <ccr@tnsp.org> parents: 189 diff changeset	433 # XXX Hashify the array into lat/long keys
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	434
190 15c49bb1ce05 Add some comments. Matti Hamalainen <ccr@tnsp.org> parents: 189 diff changeset	435 # This is horrible :S
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	436 my $fcrap = $fdata->{"om:featureOfInterest"}{"sams:SF_SpatialSamplingFeature"}{"sams:shape"}{"gml:MultiPoint"}{"gml:pointMember"};
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	437 foreach my $fnode (@{$fcrap})
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	438 {
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	439 my $floc = $fnode->{"gml:Point"};
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	440 if ($floc->{"gml:pos"} =~ /^\s([\+\-]?\d+\.\d)\s+([\+\-]?\d+\.\d)\s$/)
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	441 {
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	442 my ($flat, $flong) = ($1, $2);
190 15c49bb1ce05 Add some comments. Matti Hamalainen <ccr@tnsp.org> parents: 189 diff changeset	443 # Should use a hash -
189 3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	444 foreach my $flol (@farray)
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	445 {
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	446 if ($flol->{"lat"} == $flat && $flol->{"long"} == $flong)
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	447 {
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	448 $weatherdata->{$floc->{"gml:name"}} = [
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	449 $flol->{"time"},
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	450 $flol->{"temp"},
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	451 "",
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	452 "",
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	453 defined($flol->{"humidity"}) ? $flol->{"humidity"} : "",
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	454 ];
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	455 }
3af8ea19a3e2 Improved weather fetching. Matti Hamalainen <ccr@tnsp.org> parents: 186 diff changeset	456 }
169 c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	457 }
c12e422a543a FMI data parsing kinda almost works. It is horrible, though. XML ... :Q Matti Hamalainen <ccr@tnsp.org> parents: 168 diff changeset	458 }
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	459 }
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	460 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	461
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	462
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	463 ###
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	464 ### Output
178 9b8ec700ede4 Clean up the weather data parser backend a bit. Matti Hamalainen <ccr@tnsp.org> parents: 170 diff changeset	465 ###
164 d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	466 if (opt_chk_valid("outfile", 1)) {
d5a0e4248f3e weather: Implement configuration file support for weather fetcher backend. Matti Hamalainen <ccr@tnsp.org> parents: 162 diff changeset	467 open(STDOUT, '>', opt_get("outfile")) or die("Could not open output file '".opt_get("outfile")."'.\n");
158 db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	468 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	469
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	470 binmode STDOUT, ':encoding(utf-8)';
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	471
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	472 foreach my $key (sort { $a cmp $b } keys %$weatherdata)
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	473 {
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	474 print STDOUT $key."\|".join("\|", @{$weatherdata->{$key}})."\n";
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	475 }
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	476
db2b1f74d994 Initial import of weather data fetching backend, written in Perl. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	477 close(STDOUT);

Mercurial > hg > egg-tcls

annotate fetch_weather.pl @ 195:9eee4e1b757c