Mercurial > hg > egg-tcls

annotate convert_feeds_db.tcl @ 178:9b8ec700ede4

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Clean up the weather data parser backend a bit.
author Matti Hamalainen <ccr@tnsp.org>
date Tue, 03 Jun 2014 15:10:47 +0300
parents 4c51eeba993f
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
134
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
1 #!/usr/bin/tclsh
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
2 # TCL script for converting Feed databases from
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
3 # flatfile format to SQLite3 database.
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
4 #
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
5 # Written by Matti 'ccr' Hamalainen <ccr@tnsp.org>
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
6 # (C) Copyright 2011 Tecnic Software productions (TNSP)
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
7 #
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
8 package require sqlite3
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
9 source [file dirname [info script]]/util_convert.tcl
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
10
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
11 ### Check commandline arguments
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
12 if {$argc != 2} {
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
13 puts "Usage: $argv0 <input_flat_file_db> <output_sqlite_db_file>"
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
14 exit 0
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
15 }
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
16
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
17 set db_input [lindex $argv 0]
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
18 set db_output [lindex $argv 1]
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
19
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
20 ### Ask for confirmation
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
21 puts "Conversion of '$db_input' to SQLite database '$db_output'."
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
22 puts "NOTICE! This WILL destroy the current data in the target database!"
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
23
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
24 if {![confirm_yesno "Proceed"]} {
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
25 exit 0
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
26 }
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
27
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
28 ### Open flatfile for reading
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
29 if {[catch {set fd [open $db_input r]} uerrmsg]} {
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
30 puts "Could not open file '$db_input' for reading: $uerrmsg"
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
31 exit 1
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
32 }
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
33
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
34 ### Open SQLite database, drop old tables, create new
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
35 open_db $db_output
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
36
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
37 drop_table_feeds
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
38 create_table_feeds
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
39
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
40
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
41 ### Convert data
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
42 puts -nonewline "Converting database, please wait "
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
43 set nline 0
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
44 seek $fd 0 start
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
45
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
46 while {![eof $fd]} {
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
47 incr nline
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
48 gets $fd uline
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
49 set items [split $uline "½"]
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
50 if {[llength $items] == 4} {
142
4c51eeba993f Rename table.
Matti Hamalainen <ccr@tnsp.org>
parents: 134
diff changeset
51 set sql "INSERT INTO feeds (feed,utime,url,title) VALUES ('[lescape $items 1]', [lindex $items 0], '[lescape $items 2]', '[lescape $items 3]')"
134
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
52 if {[catch {dbh eval $sql} uerrmsg]} {
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
53 puts "\nError ($nline): $uerrmsg on:\n$sql"
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
54 exit 15
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
55 }
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
56 }
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
57
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
58 if {[expr $nline % 10] == 1} {
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
59 puts -nonewline "."
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
60 flush stdout
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
61 }
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
62 }
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
63
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
64 dbh close
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
65 close $fd
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
66
089b59f94067 Add creation and conversion utilities for SQLite3 format feeds databases.
Matti Hamalainen <ccr@tnsp.org>
parents:
diff changeset
67 puts "DONE."