Mercurial > hg > egg-tcls

annotate convert_spede_mn_tuksu.tcl @ 178:9b8ec700ede4

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Clean up the weather data parser backend a bit.
author Matti Hamalainen <ccr@tnsp.org>
date Tue, 03 Jun 2014 15:10:47 +0300
parents 7b03971c6d28
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
24
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
1 #!/usr/bin/tclsh
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
2 # TCL script for converting Spede, MN and Tuksu databases from
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
3 # flatfile format to SQLite3 database.
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
4 #
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
5 # Written by Matti 'ccr' Hamalainen <ccr@tnsp.org>
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
6 # (C) Copyright 2011 Tecnic Software productions (TNSP)
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
7 #
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
8 package require sqlite3
34
dccd6c47f9cd Use special 'source' statement to include util_convert.tcl from same directory where the script calling it resides.
Matti Hamalainen <ccr@tnsp.org>
parents: 33
diff changeset
9 source [file dirname [info script]]/util_convert.tcl
24
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
10
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
11 ### Check commandline arguments
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
12 if {$argc != 3} {
63
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
13 puts "Usage: $argv0 <input_flat_file_db> <output_sqlite_db_file> <table_name>"
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
14 exit 0
24
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
15 }
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
16
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
17 set db_input [lindex $argv 0]
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
18 set db_output [lindex $argv 1]
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
19 set db_table [lindex $argv 2]
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
20
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
21 ### Ask for confirmation
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
22 puts "Conversion of '$db_input' to SQLite database '$db_output', to table '$db_table'."
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
23 puts "NOTICE! This WILL destroy the current data in table '$db_table'!"
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
24
33
8a6bfcf1f57b Use the functions that were implemented in the utility library. Cleanups.
Matti Hamalainen <ccr@tnsp.org>
parents: 24
diff changeset
25 if {![confirm_yesno "Proceed"]} {
63
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
26 exit 0
24
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
27 }
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
28
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
29 ### Open flatfile for reading
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
30 if {[catch {set fd [open $db_input r]} uerrmsg]} {
63
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
31 puts "Could not open file '$db_input' for reading: $uerrmsg"
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
32 exit 1
24
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
33 }
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
34
33
8a6bfcf1f57b Use the functions that were implemented in the utility library. Cleanups.
Matti Hamalainen <ccr@tnsp.org>
parents: 24
diff changeset
35 ### Open SQLite database, drop old tables, create new
8a6bfcf1f57b Use the functions that were implemented in the utility library. Cleanups.
Matti Hamalainen <ccr@tnsp.org>
parents: 24
diff changeset
36 open_db $db_output
8a6bfcf1f57b Use the functions that were implemented in the utility library. Cleanups.
Matti Hamalainen <ccr@tnsp.org>
parents: 24
diff changeset
37 drop_table $db_table
36
b24cb1dd0423 Fail under certain conditions only when creating tables.
Matti Hamalainen <ccr@tnsp.org>
parents: 34
diff changeset
38 create_table_quotes_votes $db_table 1
24
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
39
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
40 ### Convert data
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
41 puts -nonewline "Converting database, please wait "
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
42 set nline 0
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
43 seek $fd 0 start
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
44 while {![eof $fd]} {
63
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
45 incr nline
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
46 set line [gets $fd]
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
47 if {$line != ""} {
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
48 set items [split $line "|"]
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
49 set host [lindex $items 3]
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
50 set uid [lindex $items 4]
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
51 set sql "INSERT INTO $db_table (id,utime,utext,user,host,chan) VALUES ([lindex $items 0], [lindex $items 2], '[lescape $items 1]', '[lescape $items 3]', '[lescape $items 4]', '[lescape $items 5]')"
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
52 if {[catch {dbh eval $sql} uerrmsg]} {
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
53 puts "\nError ($nline): $uerrmsg on:\n$sql"
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
54 exit 15
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
55 }
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
56 }
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
57 if {[expr $nline % 10] == 1} {
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
58 puts -nonewline "."
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
59 flush stdout
7b03971c6d28 Remove tabs and reindent.
Matti Hamalainen <ccr@tnsp.org>
parents: 36
diff changeset
60 }
24
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
61 }
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
62
33
8a6bfcf1f57b Use the functions that were implemented in the utility library. Cleanups.
Matti Hamalainen <ccr@tnsp.org>
parents: 24
diff changeset
63 dbh close
24
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
64 close $fd
ac38d12e3b72 Add conversion script for Spede, MN and Tuksu style quote databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
65
33
8a6bfcf1f57b Use the functions that were implemented in the utility library. Cleanups.
Matti Hamalainen <ccr@tnsp.org>
parents: 24
diff changeset
66 puts "DONE."