egg-tcls: urllog.tcl annotate

annotate urllog.tcl @ 594:a488be0f45e0

urllog: More cleanups.

author	Matti Hamalainen <ccr@tnsp.org>
date	Thu, 28 Jan 2021 12:47:20 +0200
parents	ac72a228089b
children	7f7d8048ecb5

rev	line source
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	1 ##########################################################################
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	2 #
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	3 # URLLog v2.7.0 by Matti 'ccr' Hamalainen <ccr@tnsp.org>
578 14dfb925a64a Bump copyright years. Matti Hamalainen <ccr@tnsp.org> parents: 573 diff changeset	4 # (C) Copyright 2000-2021 Tecnic Software productions (TNSP)
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	5 #
113 077c7383f36f urllog: Add line about the script's license. Matti Hamalainen <ccr@tnsp.org> parents: 112 diff changeset	6 # This script is freely distributable under GNU GPL (version 2) license.
077c7383f36f urllog: Add line about the script's license. Matti Hamalainen <ccr@tnsp.org> parents: 112 diff changeset	7 #
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	8 ##########################################################################
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	9 #
591 b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	10 # URL-logger script for EggDrop IRC robot, utilizing TCL SQLite3 database
b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	11 # interface. Requirements for this script are as follows:
458 cfbe6acc1d73 urllog: tcl-tls 1.7.x (tested with 1.7.13) is now a requirement. It is Matti Hamalainen <ccr@tnsp.org> parents: 457 diff changeset	12 #
591 b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	13 # - utillib.tcl (available from same repository as this script)
b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	14 # - tcl-tls 1.7.13+ for SSL/TLS support
b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	15 # - TCL 8.6 or later
b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	16 # - Eggdrop 1.6.20 or later
b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	17 # - SQLite3 and TCL bindings for it.
b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	18 #
b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	19 # On Debian: tcl8.6 tcl-tls libsqlite3-tcl eggdrop eggdrop-data
50 f69363fc1f61 Update some comments and add a bit of documentation. Matti Hamalainen <ccr@tnsp.org> parents: 49 diff changeset	20 #
479 fc3b6bc37927 urllog: Improve documentation slightly. Matti Hamalainen <ccr@tnsp.org> parents: 473 diff changeset	21 # If you are doing a fresh install, you will need to create the initial
fc3b6bc37927 urllog: Improve documentation slightly. Matti Hamalainen <ccr@tnsp.org> parents: 473 diff changeset	22 # database with the required table schemas. You can do that by running
503 fdd1f0b83685 urllog: Migration from old versions hasn't been supported in reality for a Matti Hamalainen <ccr@tnsp.org> parents: 493 diff changeset	23 # "create_urllog_db.tcl". You also need to set up the configuration in
fdd1f0b83685 urllog: Migration from old versions hasn't been supported in reality for a Matti Hamalainen <ccr@tnsp.org> parents: 493 diff changeset	24 # "config.urllog" file.
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	25 #
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	26 ##########################################################################
13 e06d41fb69d5 Begin work on converting urllog.tcl to use an SQLite3 database instead of flat file. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	27
263 f01d60175c44 urllog: Move configuration to external file. Matti Hamalainen <ccr@tnsp.org> parents: 260 diff changeset	28 ### The configuration should be in config.urllog in same directory
f01d60175c44 urllog: Move configuration to external file. Matti Hamalainen <ccr@tnsp.org> parents: 260 diff changeset	29 ### as this script. Or change the line below to point where ever
f01d60175c44 urllog: Move configuration to external file. Matti Hamalainen <ccr@tnsp.org> parents: 260 diff changeset	30 ### you wish. See "config.urllog.example" for an example config file.
f01d60175c44 urllog: Move configuration to external file. Matti Hamalainen <ccr@tnsp.org> parents: 260 diff changeset	31 source [file dirname [info script]]/config.urllog
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	32
291 54d34d086b47 urllog: Use the utility lib for entity conversion. Matti Hamalainen <ccr@tnsp.org> parents: 289 diff changeset	33 ### Required utillib.tcl
54d34d086b47 urllog: Use the utility lib for entity conversion. Matti Hamalainen <ccr@tnsp.org> parents: 289 diff changeset	34 source [file dirname [info script]]/utillib.tcl
54d34d086b47 urllog: Use the utility lib for entity conversion. Matti Hamalainen <ccr@tnsp.org> parents: 289 diff changeset	35
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	36
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	37 ##########################################################################
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	38 # No need to look below this line
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	39 ##########################################################################
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	40 package require sqlite3
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	41 package require http
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	42
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	43 set urllog_name "URLLog"
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	44 set urllog_version "2.7.0"
578 14dfb925a64a Bump copyright years. Matti Hamalainen <ccr@tnsp.org> parents: 573 diff changeset	45 set urllog_message "$urllog_name v$urllog_version (C) 2000-2021 ccr/TNSP"
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	46
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	47
424 825cac46b1cb Cosmetic / stray trailing whitespace cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 422 diff changeset	48 set urllog_httprep [split "\@\|%40\|{\|%7B\|}\|%7D\|\[\|%5B\|\]\|%5D" "\|"]
592 ac72a228089b urllog: Rename urllog_shorturl_str to urllog_shorturl_codes. Matti Hamalainen <ccr@tnsp.org> parents: 591 diff changeset	49 set urllog_shorturl_codes "ABCDEFGHIJKLNMOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	50
102 5425dc418505 urllog: Entity data is now in UTF-8, but TCL source files are interpreted with current system locale, which may not be UTF-8. We must therefore "convert" the entity mapping string to UTF-8 to be certain of TCL's interpretation of its encoding. Matti Hamalainen <ccr@tnsp.org> parents: 101 diff changeset	51
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	52 #-------------------------------------------------------------------------
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	53 ### Utility functions
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	54 proc urllog_log {umsg} {
267 da239a953e24 urllog: Change some setting names, etc. Matti Hamalainen <ccr@tnsp.org> parents: 264 diff changeset	55 global urllog_log_enable urllog_name
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	56
267 da239a953e24 urllog: Change some setting names, etc. Matti Hamalainen <ccr@tnsp.org> parents: 264 diff changeset	57 if {$urllog_log_enable != 0} {
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	58 putlog "$urllog_name: $umsg"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	59 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	60 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	61
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	62
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	63 proc urllog_qm {uid} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	64 global urllog_messages
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	65
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	66 if {[info exists urllog_messages($uid)]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	67 return $urllog_messages($uid)
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	68 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	69 return $uid
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	70 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	71 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	72
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	73
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	74 proc urllog_smsg {apublic anick achan amsg {aargs {}}} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	75 global urllog_preferredmsg urllog_cmd_name
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	76 set amsg [string map [list "@cmd@" $urllog_cmd_name] $amsg]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	77 utl_msg_args $urllog_preferredmsg $apublic $anick $achan $amsg $aargs
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	78 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	79
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	80
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	81 proc urllog_msg {apublic anick achan aid {aargs {}}} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	82 urllog_smsg $apublic $anick $achan [urllog_qm $aid] $aargs
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	83 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	84
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	85
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	86 proc urllog_verb_msg {apublic anick achan aid {aargs {}}} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	87 global urllog_verbose
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	88 if {$urllog_verbose != 0} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	89 urllog_msg $apublic $anick $achan $aid $aargs
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	90 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	91 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	92
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	93
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	94 proc urllog_isnumber {uarg} {
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	95 foreach i [split $uarg {}] {
65 31c8c4f50aa6 urllog: Improve urllog_isnumber function. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	96 if {![string match \[0-9\] $i]} { return 0 }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	97 }
65 31c8c4f50aa6 urllog: Improve urllog_isnumber function. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	98 return 1
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	99 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	100
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	101
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	102 proc urllog_is_enabled {uval} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	103 if {$uval} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	104 return "ON."
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	105 } else {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	106 return "OFF."
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	107 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	108 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	109
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	110
116 4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	111 proc urllog_sanitize_encoding {uencoding} {
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	112 regsub -- "^\[a-z\]\[a-z\]_\[A-Z\]\[A-Z\]\." $uencoding "" uencoding
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	113 set uencoding [string tolower $uencoding]
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	114 regsub -- "^iso-" $uencoding "iso" uencoding
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	115 return $uencoding
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	116 }
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	117
294 2bb9bcfb104a Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 291 diff changeset	118
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	119 proc urllog_get_ss {uindex} {
592 ac72a228089b urllog: Rename urllog_shorturl_str to urllog_shorturl_codes. Matti Hamalainen <ccr@tnsp.org> parents: 591 diff changeset	120 global urllog_shorturl_codes
ac72a228089b urllog: Rename urllog_shorturl_str to urllog_shorturl_codes. Matti Hamalainen <ccr@tnsp.org> parents: 591 diff changeset	121 return [string index $urllog_shorturl_codes $uindex]
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	122 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	123
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	124
150 52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	125 proc urllog_get_short {utime} {
592 ac72a228089b urllog: Rename urllog_shorturl_str to urllog_shorturl_codes. Matti Hamalainen <ccr@tnsp.org> parents: 591 diff changeset	126 global urllog_shorturl_prefix urllog_shorturl_codes
150 52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	127
592 ac72a228089b urllog: Rename urllog_shorturl_str to urllog_shorturl_codes. Matti Hamalainen <ccr@tnsp.org> parents: 591 diff changeset	128 set ulen [string length $urllog_shorturl_codes]
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	129
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	130 set u1 [expr $utime / ($ulen * $ulen)]
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	131 set utmp [expr $utime % ($ulen * $ulen)]
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	132 set u2 [expr $utmp / $ulen]
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	133 set u3 [expr $utmp % $ulen]
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	134
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	135 return "\[ $urllog_shorturl_prefix[urllog_get_ss $u1][urllog_get_ss $u2][urllog_get_ss $u3] \]"
150 52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	136 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	137
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	138
594 a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	139 proc urllog_chop_str {ustr umax} {
a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	140 if {[string length $ustr] > $umax} {
a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	141 return "[string range $ustr 0 $umax]..."
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	142 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	143 return $ustr
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	144 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	145 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	146
241 669842725e2f Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 240 diff changeset	147
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	148 #-------------------------------------------------------------------------
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	149 proc urllog_add_url {urlStr urlNick urlHost urlChan urlTitle} {
594 a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	150 global urllog_db urllog_shorturl_enable urllog_pub_channels
a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	151 global urllog_add_title_max urllog_add_url_max
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	152
18 1e2232135354 More changes for SQLite support. Matti Hamalainen <ccr@tnsp.org> parents: 13 diff changeset	153
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	154 ### Does the URL already exist?
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	155 set usql "SELECT id AS uid, utime AS utime, url AS uurl, user AS uuser, host AS uhost, chan AS uchan, title AS utitle FROM urls WHERE url='[utl_escape $urlStr]'"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	156 urllog_db eval $usql {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	157 urllog_log "URL said by $urlNick ($urlStr) already known"
83 f171a9fb7b7b urllog: Split urllog_add function to urllog_exists for checking whether given URL already exists in the database. Use urllog_exists where appropriate. Matti Hamalainen <ccr@tnsp.org> parents: 82 diff changeset	158
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	159 if {[utl_match_delim_list $urllog_pub_channels $uchan]} {
594 a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	160 if {$urllog_shorturl_enable != 0} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	161 urllog_verb_msg 1 $urlNick $urlChan "url_known_short" [list $uuser $uchan $uhost [utl_ctime $utime] $utitle $uurl [urllog_get_short $uid]]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	162 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	163 urllog_verb_msg 1 $urlNick $urlChan "url_known_long" [list $uuser $uchan $uhost [utl_ctime $utime] $utitle $uurl]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	164 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	165 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	166 return 1
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	167 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	168
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	169 ### Validate title
93 4e02c0219afe urllog: Insert NULL into title column when we didn't get a title. Matti Hamalainen <ccr@tnsp.org> parents: 92 diff changeset	170 if {$urlTitle == ""} {
4e02c0219afe urllog: Insert NULL into title column when we didn't get a title. Matti Hamalainen <ccr@tnsp.org> parents: 92 diff changeset	171 set uins "NULL"
4e02c0219afe urllog: Insert NULL into title column when we didn't get a title. Matti Hamalainen <ccr@tnsp.org> parents: 92 diff changeset	172 } else {
295 141bb4a2b76f utillib: utl_escape (which will be deprecated soon). Matti Hamalainen <ccr@tnsp.org> parents: 294 diff changeset	173 set uins "'[utl_escape $urlTitle]'"
93 4e02c0219afe urllog: Insert NULL into title column when we didn't get a title. Matti Hamalainen <ccr@tnsp.org> parents: 92 diff changeset	174 }
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	175
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	176 ### Attempt to insert into database
295 141bb4a2b76f utillib: utl_escape (which will be deprecated soon). Matti Hamalainen <ccr@tnsp.org> parents: 294 diff changeset	177 set usql "INSERT INTO urls (utime,url,user,host,chan,title) VALUES ([unixtime], '[utl_escape $urlStr]', '[utl_escape $urlNick]', '[utl_escape $urlHost]', '[utl_escape $urlChan]', $uins)"
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	178 if {[catch {urllog_db eval $usql} uerrmsg]} {
83 f171a9fb7b7b urllog: Split urllog_add function to urllog_exists for checking whether given URL already exists in the database. Use urllog_exists where appropriate. Matti Hamalainen <ccr@tnsp.org> parents: 82 diff changeset	179 urllog_log "$uerrmsg on SQL:\n$usql"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	180 return 0
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	181 }
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	182 set uid [urllog_db last_insert_rowid]
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	183 urllog_log "Added URL ($urlNick@$urlChan): $urlStr"
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	184
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	185
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	186 ### Let's say something, to confirm that everything went well.
594 a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	187 if {$urllog_shorturl_enable != 0} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	188 set urlShort [urllog_get_short $uid]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	189 set ushort "short"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	190 } else {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	191 set urlShort ""
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	192 set ushort "long"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	193 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	194
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	195 if {[string length $urlTitle] > 0} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	196 set umode "url_added_${ushort}_has_title"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	197 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	198 set umode "url_added_${ushort}_no_title"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	199 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	200
594 a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	201 urllog_verb_msg 1 $urlNick $urlChan $umode [list $urlTitle [urllog_chop_str $urlTitle $urllog_add_title_max] $urlStr [urllog_chop_str $urlStr $urllog_add_url_max] $urlShort]
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	202 return 1
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	203 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	204
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	205
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	206 #-------------------------------------------------------------------------
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	207 proc urllog_clear_request { urlStatus urlSCode urlCode urlData urlMeta } {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	208
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	209 ### Clear the request data
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	210 upvar $urlStatus ustatus
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	211 upvar $urlSCode uscode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	212 upvar $urlCode ucode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	213 upvar $urlData udata
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	214 upvar $urlMeta umeta
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	215
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	216 unset ustatus
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	217 unset uscode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	218 unset ucode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	219 unset udata
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	220 array unset umeta
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	221 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	222
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	223
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	224 #-------------------------------------------------------------------------
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	225 proc urllog_do_request { urlNick urlChan urlStr urlStatus urlSCode urlCode urlData urlMeta } {
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	226
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	227 upvar $urlStatus ustatus
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	228 upvar $urlSCode uscode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	229 upvar $urlCode ucode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	230 upvar $urlData udata
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	231 upvar $urlMeta umeta
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	232
456 102dc89488af urllog: Improve how http headers are formed. Matti Hamalainen <ccr@tnsp.org> parents: 425 diff changeset	233 set urlHeaders {}
102dc89488af urllog: Improve how http headers are formed. Matti Hamalainen <ccr@tnsp.org> parents: 425 diff changeset	234 lappend urlHeaders "Accept-Encoding" "identity"
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	235 #lappend urlHeaders "Connection" "keep-alive"
456 102dc89488af urllog: Improve how http headers are formed. Matti Hamalainen <ccr@tnsp.org> parents: 425 diff changeset	236
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	237 ### Perform request
456 102dc89488af urllog: Improve how http headers are formed. Matti Hamalainen <ccr@tnsp.org> parents: 425 diff changeset	238 if {[catch {set utoken [::http::geturl $urlStr -timeout 6000 -binary 1 -headers $urlHeaders]} uerrmsg]} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	239 urllog_verb_msg 1 $urlNick $urlChan "err_http_get" [list $urlStr $uerrmsg]
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	240 urllog_log "HTTP request failed: $uerrmsg"
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	241 return 0
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	242 }
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	243
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	244 ### Check status
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	245 set ustatus [::http::status $utoken]
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	246 set uscode [::http::code $utoken]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	247 set ucode [::http::ncode $utoken]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	248
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	249 if {$ustatus != "ok"} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	250 urllog_verb_msg 1 $urlNick $urlChan "err_http_status" [list $urlStr $ustatus $uscode $ucode]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	251 urllog_log "Error in HTTP request: $ustatus / $uscode ($urlStr)"
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	252 return 0
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	253 }
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	254
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	255 ### Get data
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	256 set udata [::http::data $utoken]
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	257 array set umeta [::http::meta $utoken]
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	258 ::http::cleanup $utoken
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	259
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	260 ### Sanitize the metadata KEYS
584 9b64f201b3a7 urllog: Use lowercase HTTP metadata keys and sanitize them. Matti Hamalainen <ccr@tnsp.org> parents: 583 diff changeset	261 foreach {ukey uvalue} [array get umeta] {
9b64f201b3a7 urllog: Use lowercase HTTP metadata keys and sanitize them. Matti Hamalainen <ccr@tnsp.org> parents: 583 diff changeset	262 set ukey [string tolower $ukey]
9b64f201b3a7 urllog: Use lowercase HTTP metadata keys and sanitize them. Matti Hamalainen <ccr@tnsp.org> parents: 583 diff changeset	263 set umeta($ukey) $uvalue
9b64f201b3a7 urllog: Use lowercase HTTP metadata keys and sanitize them. Matti Hamalainen <ccr@tnsp.org> parents: 583 diff changeset	264 }
9b64f201b3a7 urllog: Use lowercase HTTP metadata keys and sanitize them. Matti Hamalainen <ccr@tnsp.org> parents: 583 diff changeset	265
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	266 return 1
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	267 }
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	268
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	269
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	270 #-------------------------------------------------------------------------
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	271 proc urllog_validate_url { urlNick urlChan urlMStr urlMProto urlMHostName } {
594 a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	272 global urllog_httprep urllog_shorturl_prefix urllog_shorturl_enable
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	273
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	274 upvar $urlMStr urlStr
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	275 upvar $urlMProto urlProto
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	276 upvar $urlMHostName urlHostName
3 8003090caa35 Lots of code cleanups, add "fixer" for RasiaTube links (which suck) to point directly to Youtube. Matti Hamalainen <ccr@tnsp.org> parents: 0 diff changeset	277
571 d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	278 ### Hack for removing parenthesis around an URL
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	279 if {[regexp {^$(.+)$$} $urlStr urlMatch urlClean]} {
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	280 set urlStr $urlClean
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	281 }
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	282
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	283 if {[regexp {^\[(.+)\]$} $urlStr urlMatch urlClean]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	284 set urlStr $urlClean
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	285 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	286
571 d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	287 ### Clean excess stuff, if any, and attempt to
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	288 ### guess the URL protocol component if it is missing
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	289 if {[regexp "(\[a-z\]+)://\[^ \]+" $urlStr urlMatch urlProto]} {
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	290 set urlStr $urlMatch
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	291 } elseif {[regexp "www\.\[^ \]+" $urlStr urlMatch]} {
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	292 set urlStr "http://$urlMatch"
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	293 } elseif {[regexp "ftp\.\[^ \]+" $urlStr urlMatch]} {
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	294 set urlStr "ftp://$urlMatch"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	295 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	296
95 687bdd74dfac urllog: Check if TLS support is enabled when checking if we can fetch title information via HTTP or SSL/HTTP. Matti Hamalainen <ccr@tnsp.org> parents: 93 diff changeset	297 ### Handle URLs that have an IPv4-address
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	298 if {[regexp "(\[a-z\]+)://(\[0-9\]{1,3})\\.(\[0-9\]{1,3})\\.(\[0-9\]{1,3})\\.(\[0-9\]{1,3})" $urlStr urlMatch urlProto ni1 ni2 ni3 ni4]} {
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	299 # Check if the IP is on local network
572 295c225e3152 urllog: Improve invalid/local IPv4 network check Matti Hamalainen <ccr@tnsp.org> parents: 571 diff changeset	300 if {$ni1 == 127 \|\| $ni1 == 10 \|\| ($ni1 == 192 && $ni2 == 168)} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	301 urllog_verb_msg 1 $urlNick $urlChan "err_url_local_net" [list $urlStr]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	302 urllog_log "URL pointing to local network, ignored (${urlStr})."
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	303 return 0
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	304 }
572 295c225e3152 urllog: Improve invalid/local IPv4 network check Matti Hamalainen <ccr@tnsp.org> parents: 571 diff changeset	305 if {$ni1 == 0 \|\| $ni1 >= 255 \|\| $ni2 >= 255 \|\| $ni3 >= 255 \|\| $ni4 >= 255} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	306 urllog_verb_msg 1 $urlNick $urlChan "err_url_invalid_net" [list $urlStr]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	307 urllog_log "URL pointing to invalid network, ignored (${urlStr})."
571 d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	308 return 0
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	309 }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	310 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	311
96 e5a6c27be365 urllog: Comments and cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 95 diff changeset	312 ### Check now if we have an ShortURL here ...
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	313 if {[string match "${urllog_shorturl_prefix}*" $urlStr]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	314 urllog_log "Ignoring ShortURL: ${urlStr}"
252 eb2fce89b8ab urllog: Comment out some currently unused code. Matti Hamalainen <ccr@tnsp.org> parents: 251 diff changeset	315 # set uud ""
eb2fce89b8ab urllog: Comment out some currently unused code. Matti Hamalainen <ccr@tnsp.org> parents: 251 diff changeset	316 # set usql "SELECT id AS uid, url AS uurl, user AS uuser, host AS uhost, chan AS uchan, title AS utitle FROM urls WHERE utime=$uud"
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	317 # urllog_db eval $usql {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	318 # urllog_msg 1 $urlNick $urlChan "'$utitle' - $uurl"
252 eb2fce89b8ab urllog: Comment out some currently unused code. Matti Hamalainen <ccr@tnsp.org> parents: 251 diff changeset	319 # return 1
eb2fce89b8ab urllog: Comment out some currently unused code. Matti Hamalainen <ccr@tnsp.org> parents: 251 diff changeset	320 # }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	321 return 0
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	322 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	323
95 687bdd74dfac urllog: Check if TLS support is enabled when checking if we can fetch title information via HTTP or SSL/HTTP. Matti Hamalainen <ccr@tnsp.org> parents: 93 diff changeset	324 ### Get URL protocol component
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	325 set urlProto ""
464 506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	326 if {[regexp "(\[a-z\]+)://" $urlStr urlMatch urlProto]} {
506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	327 ### Is it a http or ftp url?
506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	328 if {$urlProto != "http" && $urlProto != "https" && $urlProto != "ftp"} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	329 urllog_verb_msg 1 $urlNick $urlChan "err_url_proto_class" [list $urlStr $urlProto]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	330 urllog_log "Broken URL: ${urlStr} - unsupported protocol class (${urlProto})."
464 506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	331 return 0
506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	332 }
506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	333 } else {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	334 urllog_verb_msg 1 $urlNick $urlChan "err_url_proto_no_class" [list $urlStr]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	335 urllog_log "Broken URL: ${urlStr} - no protocol specifier."
464 506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	336 return 0
506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	337 }
95 687bdd74dfac urllog: Check if TLS support is enabled when checking if we can fetch title information via HTTP or SSL/HTTP. Matti Hamalainen <ccr@tnsp.org> parents: 93 diff changeset	338
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	339 ### Check the PORT (if the ":" is there)
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	340 set urlRecord [split $urlStr "/"]
a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	341 set urlHostName [lindex $urlRecord 2]
a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	342 set urlPort [lindex [split $urlHostName ":"] end]
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	343
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	344 if {$urlPort != "" && ![urllog_isnumber $urlPort] && $urlPort != $urlHostName} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	345 urllog_verb_msg 1 $urlNick $urlChan "err_url_invalid_port" [list $urlStr $urlPort]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	346 urllog_log "Broken URL: ${urlStr} - illegal or invalid port '${urlPort}'"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	347 return 0
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	348 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	349
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	350
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	351 set urlStr [string map $urllog_httprep $urlStr]
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	352 return 1
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	353 }
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	354
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	355
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	356 #-------------------------------------------------------------------------
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	357 proc urllog_handle_redirect {urlNick urlHost urlChan urlRedirLevel urlProto urlHostName urlStr urlStatus urlSCode urlCode urlData urlMeta} {
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	358
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	359 upvar $urlProto uproto
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	360 upvar $urlHostName uhostname
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	361 upvar $urlStr ustr
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	362 upvar $urlStatus ustatus
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	363 upvar $urlSCode uscode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	364 upvar $urlCode ucode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	365 upvar $urlData udata
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	366 upvar $urlMeta umeta
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	367
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	368 ### Was result a redirect?
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	369 if {$ucode >= 301 && $ucode <= 303} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	370 ### Check that we have a location header
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	371 if {![info exists umeta(location)]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	372 urllog_verb_msg 1 $urlNick $urlChan "err_redirect_invalid" [list $ustr $ustatus $uscode $ucode $urlRedirLevel]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	373 urllog_log "Invalid redirect without location header: status=$ustatus, code=$ucode, scode=$uscode, url=$ustr, redirLevel=$urlRedirLevel"
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	374 return 0
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	375 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	376
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	377 ### Fix up location URI
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	378 set nustr $umeta(location)
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	379 if {![regexp "\[a-z\]+://" $nustr]} {
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	380 if {[string range $nustr 0 0] != "/"} {
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	381 append nustr "/"
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	382 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	383 set nustr "${uproto}://${uhostname}${nustr}"
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	384 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	385
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	386 ### Validate the target URI
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	387 urllog_log "Redirection #${urlRedirLevel}: $ustr -> $nustr"
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	388 set ustr $nustr
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	389
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	390 if {![urllog_validate_url $urlNick $urlChan ustr uproto uhostname]} {
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	391 return 0
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	392 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	393
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	394 ### Attempt to fetch redirection target
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	395 urllog_clear_request ustatus uscode ucode udata umeta
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	396 if {![urllog_do_request $urlNick $urlChan $ustr ustatus uscode ucode udata umeta]} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	397 urllog_verb_msg 1 $urlNick $urlChan "err_redirect_fail" [list $ustr $ustatus $uscode $ucode $urlRedirLevel]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	398 urllog_log "Error fetching redirect: status=$ustatus, code=$ucode, scode=$uscode, url=$ustr, redirLevel=$urlRedirLevel"
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	399 return 0
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	400 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	401 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	402
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	403 return 1
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	404 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	405
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	406
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	407 #-------------------------------------------------------------------------
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	408 proc urllog_check_url {urlStr urlNick urlHost urlChan} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	409 global urllog_encoding http_tls_support
581 148c7553c50f urllog: Remove rasiatube hack setting, it's not used. Matti Hamalainen <ccr@tnsp.org> parents: 579 diff changeset	410 global urllog_extra_checks urllog_extra_strict
3 8003090caa35 Lots of code cleanups, add "fixer" for RasiaTube links (which suck) to point directly to Youtube. Matti Hamalainen <ccr@tnsp.org> parents: 0 diff changeset	411
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	412 ### Validate URL compoments, etc.
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	413 set urlProto ""
a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	414 set urlHostName ""
a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	415 if {![urllog_validate_url $urlNick $urlChan urlStr urlProto urlHostName]} {
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	416 return 1
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	417 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	418
267 da239a953e24 urllog: Change some setting names, etc. Matti Hamalainen <ccr@tnsp.org> parents: 264 diff changeset	419 ### Do we perform additional checks?
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	420 if {$urllog_extra_checks == 0 \|\| !(($http_tls_support != 0 && $urlProto == "https") \|\| $urlProto == "http")} {
230 3e3756b113a1 urllog: Cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 229 diff changeset	421 # No optional checks, or it's not http/https.
306 9858b93387a2 urllog: 100L. Matti Hamalainen <ccr@tnsp.org> parents: 304 diff changeset	422 if {$urllog_extra_strict == 0} {
304 f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	423 # Strict checking disabled, so add the URL, if it does not exist already.
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	424 urllog_add_url $urlStr $urlNick $urlHost $urlChan ""
304 f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	425 return 1
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	426 } elseif {$http_tls_support == 0 && $urlProto == "https"} {
304 f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	427 # Strict ENABLED: If TLS support is disabled and we have https, do nothing
f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	428 return 1
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	429 } elseif {$urlProto != "http" && $urlProto != "https"} {
304 f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	430 # Strict ENABLED: It's not http, or https
f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	431 return 1
f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	432 }
7 50b52294e93e urllog: Strip &rlm; entities from titles; Some work on SSL/https support. Matti Hamalainen <ccr@tnsp.org> parents: 4 diff changeset	433
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	434 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	435 ### Does the document pointed by the URL exist?
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	436 if {![urllog_do_request $urlNick $urlChan $urlStr ustatus uscode ucode udata umeta]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	437 return 1
116 4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	438 }
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	439
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	440 ### Handle redirects of 3 levels
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	441 if {![urllog_handle_redirect $urlNick $urlHost $urlChan 1 urlProto urlHostName urlStr ustatus uscode ucode udata umeta]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	442 return 1
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	443 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	444
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	445 if {![urllog_handle_redirect $urlNick $urlHost $urlChan 2 urlProto urlHostName urlStr ustatus uscode ucode udata umeta]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	446 return 1
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	447 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	448
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	449 if {![urllog_handle_redirect $urlNick $urlHost $urlChan 3 urlProto urlHostName urlStr ustatus uscode ucode udata umeta]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	450 return 1
116 4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	451 }
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	452
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	453 # Final document
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	454 if {$ucode >= 200 && $ucode <= 205} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	455 set uenc_doc ""
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	456 set uenc_http ""
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	457 set uencoding ""
116 4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	458
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	459 # Get information about specified character encodings
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	460 if {[info exists umeta(content-type)] && [regexp -nocase {charset\s=\s([a-z0-9._-]+)} $umeta(content-type) umatches uenc_http]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	461 # Found character set encoding information in HTTP headers
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	462 }
470 2faf2eb18f26 urllog: Add support for Twitter meta titles. Matti Hamalainen <ccr@tnsp.org> parents: 469 diff changeset	463
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	464 if {[regexp -nocase -- "<meta.\\?content=\"text/html.\\?charset=(\[^\"\])\".\\?/\?>" $udata umatches uenc_doc]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	465 # Found old style HTML meta tag with character set information
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	466 } elseif {[regexp -nocase -- "<meta.\\?charset=\"(\[^\"\])\".\*\?/\?>" $udata umatches uenc_doc]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	467 # Found HTML5 style meta tag with character set information
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	468 }
313 8175ef52889b urllog: Improve URL title functionality. Matti Hamalainen <ccr@tnsp.org> parents: 312 diff changeset	469
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	470 # Make sanitized versions of the encoding strings
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	471 set uenc_http2 [urllog_sanitize_encoding $uenc_http]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	472 set uenc_doc2 [urllog_sanitize_encoding $uenc_doc]
313 8175ef52889b urllog: Improve URL title functionality. Matti Hamalainen <ccr@tnsp.org> parents: 312 diff changeset	473
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	474 # Check if the document has specified encoding
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	475 # KLUDGE!
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	476 set uencoding $uenc_http2
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	477 if {$uencoding == "" && $uenc_doc2 != ""} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	478 set uencoding $uenc_doc2
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	479 } elseif {$uencoding == ""} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	480 # If _NO_ known encoding of any kind, assume the default of iso8859-1
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	481 set uencoding "iso8859-1"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	482 }
150 52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	483
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	484 urllog_log "Charsets: http='$uenc_http', doc='$uenc_doc' / sanitized http='$uenc_http2', doc='$uenc_doc2' -> '$uencoding'"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	485
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	486 # Get the document title, if any
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	487 set urlTitle ""
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	488
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	489 if {[regexp -nocase -- "<meta name=\"twitter:title\" content=\"(.\\?)\"\\s\\/\?>" $udata umatches urlTitle]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	490 # ...
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	491 } elseif {[regexp -nocase -- "<title.\\?>(.\\?)</title>" $udata umatches urlTitle]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	492 # ...
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	493 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	494
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	495 # If facebook, get meta info
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	496 if {[regexp -nocase -- "(http\|https):\/\/www.facebook.com" $urlStr]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	497 if {[regexp -nocase -- "<meta name=\"description\" content=\"(.\*\?)\"" $udata umatches urlTmp]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	498 if {$urlTitle != ""} { append urlTitle " :: " }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	499 append urlTitle $urlTmp
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	500 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	501 }
3 8003090caa35 Lots of code cleanups, add "fixer" for RasiaTube links (which suck) to point directly to Youtube. Matti Hamalainen <ccr@tnsp.org> parents: 0 diff changeset	502
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	503 # If character set conversion is required, do it now
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	504 if {$urlTitle != "" && $uencoding != ""} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	505 if {[catch {set urlTitle [encoding convertfrom $uencoding $urlTitle]} cerrmsg]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	506 urllog_verb_msg 1 $urlNick $urlChan "err_charset" [list $urlStr $urlTitle $cerrmsg]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	507 urllog_log "Error in charset conversion: $urlStr / '$urlTitle': $cerrmsg"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	508 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	509
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	510 # Convert some HTML entities to plaintext and do some cleanup
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	511 set utmp [utl_convert_html_ent $urlTitle]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	512 regsub -all "\r\|\n\|\t" $utmp " " utmp
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	513 regsub -all " *" $utmp " " utmp
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	514 set urlTitle [string trim $utmp]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	515 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	516
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	517 # Check if the URL already exists, just in case we had some redirects
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	518 urllog_add_url $urlStr $urlNick $urlHost $urlChan $urlTitle
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	519 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	520 urllog_verb_msg 1 $urlNick $urlChan "err_http_fail" [list $urlStr $ustatus $uscode $ucode]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	521 urllog_log "Error fetching document: status=$ustatus, code=$ucode, scode=$uscode, url=$urlStr"
83 f171a9fb7b7b urllog: Split urllog_add function to urllog_exists for checking whether given URL already exists in the database. Use urllog_exists where appropriate. Matti Hamalainen <ccr@tnsp.org> parents: 82 diff changeset	522 }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	523 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	524 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	525
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	526
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	527 #-------------------------------------------------------------------------
249 d98876dd9ee1 urllog: Rename a function. Matti Hamalainen <ccr@tnsp.org> parents: 241 diff changeset	528 proc urllog_check_line {unick uhost uhand uchan utext} {
219 4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	529 global urllog_log_channels
4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	530
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	531 ### Check the nick
87 97c56d1e9ce2 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 86 diff changeset	532 if {$unick == "*"} {
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	533 return 0
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	534 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	535
219 4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	536 ### Check the channel
315 7a987b22a817 urllog: Add new configuration option urllog_msg_channels. Matti Hamalainen <ccr@tnsp.org> parents: 313 diff changeset	537 if {[utl_match_delim_list $urllog_log_channels $uchan]} {
7a987b22a817 urllog: Add new configuration option urllog_msg_channels. Matti Hamalainen <ccr@tnsp.org> parents: 313 diff changeset	538 ### Do the URL checking
7a987b22a817 urllog: Add new configuration option urllog_msg_channels. Matti Hamalainen <ccr@tnsp.org> parents: 313 diff changeset	539 foreach str [split $utext " "] {
464 506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	540 if {[regexp "(\[a-z]+://\[^\[:space:\]\]+\|^(www\|ftp)\.\[^\[:space:\]\]+)" $str ulink]} {
315 7a987b22a817 urllog: Add new configuration option urllog_msg_channels. Matti Hamalainen <ccr@tnsp.org> parents: 313 diff changeset	541 urllog_check_url $str $unick $uhost $uchan
219 4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	542 }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	543 }
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	544 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	545
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	546 return 0
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	547 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	548
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	549
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	550 #-------------------------------------------------------------------------
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	551 ### Parse arguments, find and show the results
594 a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	552 proc urllog_cmd_search {unick uhand uchan utext upublic} {
a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	553 global urllog_db urllog_shorturl_enable urllog_showmax_pub urllog_showmax_priv
a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	554 global urllog_search_title_max urllog_search_url_max
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	555
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	556 if {$upublic == 0} {
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	557 set ulimit 5
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	558 } else {
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	559 set ulimit 3
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	560 }
19 9cf22053e5da Repair !urlfind functionality. Matti Hamalainen <ccr@tnsp.org> parents: 18 diff changeset	561
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	562 ### Parse the given command
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	563 urllog_log "$unick/$uhand searched URL: $utext"
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	564
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	565 set ftokens [split $utext " "]
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	566 set fpatlist ""
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	567 foreach ftoken $ftokens {
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	568 set fprefix [string range $ftoken 0 0]
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	569 set fpattern [string range $ftoken 1 end]
295 141bb4a2b76f utillib: utl_escape (which will be deprecated soon). Matti Hamalainen <ccr@tnsp.org> parents: 294 diff changeset	570 set qpattern "'%[utl_escape $fpattern]%'"
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	571
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	572 if {$fprefix == "-"} {
128 0d21b9d1d2b9 urllog: Improve search functionality. Matti Hamalainen <ccr@tnsp.org> parents: 127 diff changeset	573 lappend fpatlist "(url NOT LIKE $qpattern OR title NOT LIKE $qpattern)"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	574 } elseif {$fprefix == "%"} {
128 0d21b9d1d2b9 urllog: Improve search functionality. Matti Hamalainen <ccr@tnsp.org> parents: 127 diff changeset	575 lappend fpatlist "user LIKE $qpattern"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	576 } elseif {$fprefix == "@"} {
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	577 # foo
112 fae3dd7a8b20 urllog: Oops, a typo in variable name. Fixed. Matti Hamalainen <ccr@tnsp.org> parents: 111 diff changeset	578 } elseif {$fprefix == "+"} {
128 0d21b9d1d2b9 urllog: Improve search functionality. Matti Hamalainen <ccr@tnsp.org> parents: 127 diff changeset	579 lappend fpatlist "(url LIKE $qpattern OR title LIKE $qpattern)"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	580 } else {
295 141bb4a2b76f utillib: utl_escape (which will be deprecated soon). Matti Hamalainen <ccr@tnsp.org> parents: 294 diff changeset	581 set qpattern "'%[utl_escape $ftoken]%'"
128 0d21b9d1d2b9 urllog: Improve search functionality. Matti Hamalainen <ccr@tnsp.org> parents: 127 diff changeset	582 lappend fpatlist "(url LIKE $qpattern OR title LIKE $qpattern)"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	583 }
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	584 }
19 9cf22053e5da Repair !urlfind functionality. Matti Hamalainen <ccr@tnsp.org> parents: 18 diff changeset	585
27 6e381916b016 Some fixes in the query mechanisms of QuoteDB and URLLog. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	586 if {[llength $fpatlist] > 0} {
6e381916b016 Some fixes in the query mechanisms of QuoteDB and URLLog. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	587 set fquery "WHERE [join $fpatlist " AND "]"
6e381916b016 Some fixes in the query mechanisms of QuoteDB and URLLog. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	588 } else {
6e381916b016 Some fixes in the query mechanisms of QuoteDB and URLLog. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	589 set fquery ""
6e381916b016 Some fixes in the query mechanisms of QuoteDB and URLLog. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	590 }
68 3762c621d1c3 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 65 diff changeset	591
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	592 ### Perform SQL query and show results if any
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	593 set nresult 0
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	594 set usql "SELECT id AS uid, utime AS utime, url AS uurl, user AS uuser, host AS uhost, title AS utitle FROM urls $fquery ORDER BY utime DESC LIMIT $ulimit"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	595 urllog_db eval $usql {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	596 incr nresult
594 a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	597 if {[string length $utitle] > 0} {
a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	598 set stitle [utl_str_map_values [urllog_qm "search_result_has_title"] [list $utitle [urllog_chop_str $utitle $urllog_search_title_max]]]
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	599 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	600 set stitle [urllog_qm "search_result_no_title"]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	601 }
594 a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	602
a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	603 if {$urllog_shorturl_enable != 0 && $uid != ""} {
a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	604 urllog_msg $upublic $unick $uchan "search_result_short" [list $nresult $uuser [utl_ctime $utime] $stitle [urllog_chop_str $uurl $urllog_search_url_max] [urllog_get_short $uid]]
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	605 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	606 urllog_msg $upublic $unick $uchan "search_result_long" [list $nresult $uuser [utl_ctime $utime] $stitle $uurl]
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	607 }
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	608 }
424 825cac46b1cb Cosmetic / stray trailing whitespace cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 422 diff changeset	609
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	610 if {$nresult == 0} {
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	611 # If no URLs were found
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	612 urllog_msg $upublic $unick $uchan "search_no_match" [list $utext]
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	613 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	614
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	615 return 1
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	616 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	617
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	618
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	619 #-------------------------------------------------------------------------
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	620 ### Finding binded functions
594 a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	621 proc urllog_cmd_pub_search {unick uhost uhand uchan utext} {
219 4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	622 global urllog_search_channels
4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	623
315 7a987b22a817 urllog: Add new configuration option urllog_msg_channels. Matti Hamalainen <ccr@tnsp.org> parents: 313 diff changeset	624 if {[utl_match_delim_list $urllog_search_channels $uchan]} {
594 a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	625 return [urllog_cmd_search $unick $uhand $uchan $utext 1]
219 4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	626 }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	627 return 0
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	628 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	629
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	630
594 a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	631 proc urllog_cmd_msg_search {unick uhost uhand utext} {
a488be0f45e0 urllog: More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 592 diff changeset	632 return [urllog_cmd_search $unick $uhand "" $utext 0]
3 8003090caa35 Lots of code cleanups, add "fixer" for RasiaTube links (which suck) to point directly to Youtube. Matti Hamalainen <ccr@tnsp.org> parents: 0 diff changeset	633 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	634
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	635
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	636 #-------------------------------------------------------------------------
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	637 # Script initialization
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	638 #-------------------------------------------------------------------------
570 0e3ee1f51c80 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 563 diff changeset	639 ### Initialization messages
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	640 putlog "$urllog_message"
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	641
570 0e3ee1f51c80 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 563 diff changeset	642
0e3ee1f51c80 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 563 diff changeset	643 ### Miscellaneous init messages
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	644 putlog " - Log messages [urllog_is_enabled $urllog_log_enable]"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	645 putlog " - Verbose mode [urllog_is_enabled $urllog_verbose]"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	646 putlog " - Additional URL validity checks [urllog_is_enabled $urllog_extra_checks]"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	647 putlog " - Strict checks [urllog_is_enabled $urllog_extra_strict]"
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	648
570 0e3ee1f51c80 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 563 diff changeset	649
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	650 ### HTTP module initialization
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	651 if {[info exists http_user_agent] && $http_user_agent != ""} {
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	652 ::http::config -useragent $http_user_agent
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	653 } else {
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	654 ::http::config -useragent "$urllog_name/$urllog_version"
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	655 }
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	656
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	657 if {[info exists http_use_proxy] && $http_use_proxy != 0} {
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	658 ::http::config -proxyhost $http_proxy_host -proxyport $http_proxy_port
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	659 putlog " - Using proxy $http_proxy_host:$http_proxy_port"
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	660 }
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	661
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	662 if {[info exists http_tls_support] && $http_tls_support != 0} {
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	663 package require tls
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	664 ::http::register https 443 [list ::tls::socket -request true -require true -ssl2 false -ssl3 false -tls1 true -tls1.1 true -tls1.2 true -cadir $http_tls_cadir -autoservername true]
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	665 putlog " - TLS/SSL support enabled."
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	666 }
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	667
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	668
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	669 ### SQLite database initialization
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	670 if {[catch {sqlite3 urllog_db $urllog_db_file} uerrmsg]} {
561 bdccc83a1c22 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 560 diff changeset	671 putlog "Could not open SQLite3 database '${urllog_db_file}': ${uerrmsg}"
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	672 exit 2
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	673 }
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	674
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	675
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	676 # end of script

Mercurial > hg > egg-tcls

annotate urllog.tcl @ 594:a488be0f45e0