# HG changeset patch # User chris # Date 1248264245 -7200 # Node ID 2fdc9cc8950252f88f077335fb7d0890f1e4d5f1 # Parent 1090e21417988147d2ef7004337730c7a16aed20 sonews/1.0.0 diff -r 1090e2141798 -r 2fdc9cc89502 .hgtags --- a/.hgtags Wed Jul 01 10:48:22 2009 +0200 +++ b/.hgtags Wed Jul 22 14:04:05 2009 +0200 @@ -1,2 +1,3 @@ 42b394eda04ba06126b04e66606ff9ce769652fc oneThreadPerSocket 19130f88c6b80cbcda5626c0a49fb35b28a8e3cb sonews-0.5.0 +88025be745057c45f7b4d63e698e1c9515e3c168 sonews/1.0.0 diff -r 1090e2141798 -r 2fdc9cc89502 DEBIAN-web/README.Debian --- a/DEBIAN-web/README.Debian Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,6 +0,0 @@ -cync for Debian ---------------- - -cync is still a very early alpha version, so be careful using it. You have been warned! - - -- Jens Mühlenhoff Mon, 11 Aug 2008 17:05:23 +0200 diff -r 1090e2141798 -r 2fdc9cc89502 DEBIAN-web/compat --- a/DEBIAN-web/compat Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1 +0,0 @@ -6 diff -r 1090e2141798 -r 2fdc9cc89502 DEBIAN-web/control --- a/DEBIAN-web/control Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,15 +0,0 @@ -Source: sonews -Section: web -Priority: optional -Maintainer: Christian Lins -Homepage: http://www.sonews.org/ -Package: sonews-web -Version: 0.5.0-beta1 -Architecture: all -Depends: kitten, libjchart2d-java, sonews -Description: Webinterface for sonews - sonews is a modern Usenet server providing newsgroups via NNTP. - The lightweight servlet server kitten is used to provide an optional - configuration web interface. - This metapackage depends on all required prerequisites to run the - servlet based webinterface of sonews. diff -r 1090e2141798 -r 2fdc9cc89502 DEBIAN/control --- a/DEBIAN/control Wed Jul 01 10:48:22 2009 +0200 +++ b/DEBIAN/control Wed Jul 22 14:04:05 2009 +0200 @@ -4,12 +4,11 @@ Maintainer: Christian Lins Homepage: http://www.sonews.org/ Package: sonews -Version: 0.6.0beta +Version: 1.0.0 Architecture: all Depends: openjdk-6-jre-headless | openjdk-6-jre | sun-java6-jre | cacao | jamvm, glassfish-mail, libmysql-java -Suggests: kitten, libpg-java, mysql-server, libjchart2d-java +Suggests: sonews-web, libpg-java, mysql-server, libjchart2d-java Description: Usenet news server sonews is a modern Usenet server providing newsgroups via NNTP. A relational database backend is used to store the news data. - The lightweight servlet server kitten is used to provide an optional - configuration web interface. + The sonews-web providers a configuration web interface. diff -r 1090e2141798 -r 2fdc9cc89502 bin/sonews.sh --- a/bin/sonews.sh Wed Jul 01 10:48:22 2009 +0200 +++ b/bin/sonews.sh Wed Jul 22 14:04:05 2009 +0200 @@ -1,6 +1,7 @@ #!/bin/bash SCRIPTROOT=$(pwd) CLASSPATH=$SCRIPTROOT/lib/sonews.jar:\ +$SCRIPTROOT/lib/sonews-helpers.jar:\ $SCRIPTROOT/lib/mysql-connector-java.jar:\ $SCRIPTROOT/lib/glassfish-mail.jar:\ $SCRIPTROOT/lib/postgresql.jar diff -r 1090e2141798 -r 2fdc9cc89502 doc/sonews.xml --- a/doc/sonews.xml Wed Jul 01 10:48:22 2009 +0200 +++ b/doc/sonews.xml Wed Jul 22 14:04:05 2009 +0200 @@ -57,11 +57,15 @@ Roadmap - sonews/0.6 - Planned to implement the XPAT command for searching, correctly + sonews/1.0 + + Various minor fixes and code cleanup (Storage and Command interface for + the upcoming Plugin API). + + XPAT command for searching, correctly hashed Message-Ids and a news purging command. See Bugtracker for - issues with target sonews/0.6.x. + issues with target sonews/1.0.x. @@ -76,17 +80,14 @@ APT easily. Add the following line to /etc/apt/sources.list: - deb http://packages.xerxys.info/debian/ unstable main - + deb http://packages.xerxys.info/debian/ unstable main And add the GPG-Key for package authentification, see Xerxys Debian Repository for more details. Then force an update of your local package list: - # apt-get update - + # apt-get update To install sonews and all prerequisites issue the following command: - # apt-get install sonews - + # apt-get install sonews This method should work for all recent Debian-based distributions (Debian, Ubuntu, etc.). @@ -124,11 +125,9 @@ You will find the SQL Schema definitions in the helpers subdirectory of the source and binary distributions. You can create the tables manually using this templates or you can use the setup helper: - user@debian$ sonews setup - + user@debian$ sonews setup or on other *nix systems: - user@nix$ java -jar sonews.jar org.sonews.util.DatabaseSetup - + user@nix$ java -jar sonews.jar org.sonews.util.DatabaseSetup The tool will ask for some information about your database environment, connect to the database, create the tables and creates a default bootstrap config file called sonews.conf. @@ -157,7 +156,7 @@ If set to true every(!) data going through sonews' socket - is written to sonews.log. After a night the logfile can be + is written to sonews.log. After a high traffic night the logfile can be several gigabytes large, so be careful with this setting. @@ -165,20 +164,40 @@ sonews.hostname - Canonical name of the server instance. This variable is part of the server's -hello message to the client and used to generate Message-Ids. + + Canonical name of the server instance. This variable is part of + the server's hello message to the client and used to generate + Message-Ids. + It is highly recommended to set sonews.hostname to the full + qualified domain name (FQDN) of the host machine. + sonews.timeout - Socket timeout for client connections in seconds. + + Socket timeout for client connections in seconds. Default as + recommended in RFC3977 is 180 seconds. + sonews.port - Listening port of sonews daemon. + + Listening port of sonews daemon. This value can be overridden + with the -p command line argument. + + + + + sonews.xdaemon.host + + + Hostname or IP address of the client machine that is allowed to + use the XDAEMON command. Default: localhost + @@ -196,20 +215,37 @@ -h|-help This output -mlgw Enables the Mailinglist Gateway poller -p portnumber Port on which sonews is listening for incoming connections. - Overrides port settings in config file and database. - - + Overrides port settings in config file and database. Webinterface The package sonews-web provides an optional webinterface that can be used to review statistical information and configuration values of sonews. - sonews-web start|stop - + sonews-web start|stop The webinterface uses the the lightweight Servlet Container Kitten and is per default listening on HTTP-Port 8080 (go to http://localhost:8080/sonews). + + + Newsgroup configuration + + Currently some manual work is necessary to create a newsgroup hosted + by a sonews instance. + + + One possibility is to talk via Telnet to the sonews instance and + use the non-standard command XDAEMON. + telnet localhost 119 + XDAEMON GROUPADD local.test 0 + Please note that the XDAEMON command has restricted access and is only + available via local connections (default, can be changed with config + value sonews.xdaemon.host). + + + You can also use the web interface to create newsgroups. + + @@ -217,8 +253,7 @@ You're welcome to create patches with bugfixes or additional features. The Mercurial DSCM makes this step an easy task. Just clone the public Mercurial repository: - hg clone http://code.xerxys.info:8000/hg/sonews/trunk sonews-trunk - + hg clone http://code.xerxys.info:8000/hg/sonews/ sonews-trunk Then make your changes, create a bundle of changesets and send this to me via email. Or ask for push access to the public repository. diff -r 1090e2141798 -r 2fdc9cc89502 helpers/commands.list --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/helpers/commands.list Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,15 @@ +org.sonews.daemon.command.ArticleCommand +org.sonews.daemon.command.CapabilitiesCommand +org.sonews.daemon.command.GroupCommand +org.sonews.daemon.command.HelpCommand +org.sonews.daemon.command.ListCommand +org.sonews.daemon.command.ListGroupCommand +org.sonews.daemon.command.ModeReaderCommand +org.sonews.daemon.command.NewGroupsCommand +org.sonews.daemon.command.NextPrevCommand +org.sonews.daemon.command.OverCommand +org.sonews.daemon.command.PostCommand +org.sonews.daemon.command.QuitCommand +org.sonews.daemon.command.StatCommand +org.sonews.daemon.command.XDaemonCommand +org.sonews.daemon.command.XPatCommand \ No newline at end of file diff -r 1090e2141798 -r 2fdc9cc89502 helpers/database_postgresql8_tmpl.sql --- a/helpers/database_postgresql8_tmpl.sql Wed Jul 01 10:48:22 2009 +0200 +++ b/helpers/database_postgresql8_tmpl.sql Wed Jul 22 14:04:05 2009 +0200 @@ -69,10 +69,11 @@ group_id INTEGER REFERENCES groups(group_id) ON DELETE CASCADE, listaddress VARCHAR(255), - PRIMARY KEY(group_id, listaddress), - UNIQUE(listaddress) + PRIMARY KEY(group_id, listaddress) ); +CREATE INDEX listaddress_key ON groups2list USING btree(listaddress); + /* Configuration table, containing key/value pairs diff -r 1090e2141798 -r 2fdc9cc89502 helpers/sonews --- a/helpers/sonews Wed Jul 01 10:48:22 2009 +0200 +++ b/helpers/sonews Wed Jul 22 14:04:05 2009 +0200 @@ -1,6 +1,7 @@ #!/bin/bash CLASSPATH=/usr/share/java/sonews.jar:\ +/usr/share/java/sonews-helpers.jar:\ /usr/share/java/mysql-connector-java.jar:\ /usr/share/java/glassfish-mail.jar:\ /usr/share/java/postgresql.jar @@ -9,7 +10,7 @@ PIDFILE=/var/run/sonews.pid ARGS="-mlgw -c /etc/sonews/sonews.conf -feed" -MAINCLASS=org.sonews.daemon.Main +MAINCLASS=org.sonews.Main JAVA=java case "$1" in @@ -31,12 +32,18 @@ done echo "done." ;; + restart) + $0 stop && $0 start + ;; setup) $JAVA -classpath $CLASSPATH org.sonews.util.DatabaseSetup ;; purge) $JAVA -classpath $CLASSPATH org.sonews.util.Purger ;; + version) + $JAVA -classpath $CLASSPATH $MAINCLASS -version + ;; *) echo "Usage: sonews [start|stop|restart|setup|purge]" esac diff -r 1090e2141798 -r 2fdc9cc89502 helpers/usage --- a/helpers/usage Wed Jul 01 10:48:22 2009 +0200 +++ b/helpers/usage Wed Jul 22 14:04:05 2009 +0200 @@ -5,4 +5,5 @@ -feed Enables feed daemon for pulling news from peer servers -h|-help This output -mlgw Enables the Mailinglist Gateway poller - -useaux Enables an additional secondary port for listening + -p Forces sonews to listen on the specified port. + -v|-version Prints out the version info an exits. diff -r 1090e2141798 -r 2fdc9cc89502 makedeb --- a/makedeb Wed Jul 01 10:48:22 2009 +0200 +++ b/makedeb Wed Jul 22 14:04:05 2009 +0200 @@ -7,11 +7,10 @@ # Create JAR files; this cannot be done with SCons, # because Scons looses inner classes. jar -cf sonews.jar -C classes/ org/ -jar -ufe sonews.jar org.sonews.daemon.Main +jar -ufe sonews.jar org.sonews.Main jar -cf test.jar -C classes/ test/ jar -ufe test.jar test.TestBench jar -cf sonews-helpers.jar helpers/ -jar -uf sonews.jar org/sonews/web/tmpl/*.tmpl # Create faked root for packaging sudo rm -r $PACKAGE_ROOT/ @@ -33,13 +32,5 @@ sudo rm -r $PACKAGE_ROOT rm -r classes/ -# Create metapackage sonews-web -PACKAGE_ROOT=sonews-web -mkdir $PACKAGE_ROOT -cp -r DEBIAN-web $PACKAGE_ROOT/DEBIAN -dpkg-deb --build $PACKAGE_ROOT -rm -r $PACKAGE_ROOT - # Check debs lintian sonews.deb -lintian sonews-web.deb diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/Main.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/Main.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,176 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews; + +import java.sql.Driver; +import java.sql.DriverManager; +import java.util.Enumeration; +import java.util.Date; +import org.sonews.config.Config; +import org.sonews.daemon.ChannelLineBuffers; +import org.sonews.daemon.Connections; +import org.sonews.daemon.NNTPDaemon; +import org.sonews.feed.FeedManager; +import org.sonews.mlgw.MailPoller; +import org.sonews.storage.StorageBackendException; +import org.sonews.storage.StorageManager; +import org.sonews.storage.StorageProvider; +import org.sonews.util.Log; +import org.sonews.util.Purger; +import org.sonews.util.io.Resource; + +/** + * Startup class of the daemon. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public final class Main +{ + + private Main() + { + } + + /** Version information of the sonews daemon */ + public static final String VERSION = "sonews/1.0.0"; + public static final Date STARTDATE = new Date(); + + /** + * The main entrypoint. + * @param args + * @throws Exception + */ + public static void main(String[] args) throws Exception + { + System.out.println(VERSION); + Thread.currentThread().setName("Mainthread"); + + // Command line arguments + boolean feed = false; // Enable feeding? + boolean mlgw = false; // Enable Mailinglist gateway? + int port = -1; + + for(int n = 0; n < args.length; n++) + { + if(args[n].equals("-c") || args[n].equals("-config")) + { + Config.inst().set(Config.LEVEL_CLI, Config.CONFIGFILE, args[++n]); + System.out.println("Using config file " + args[n]); + } + else if(args[n].equals("-dumpjdbcdriver")) + { + System.out.println("Available JDBC drivers:"); + Enumeration drvs = DriverManager.getDrivers(); + while(drvs.hasMoreElements()) + { + System.out.println(drvs.nextElement()); + } + return; + } + else if(args[n].equals("-feed")) + { + feed = true; + } + else if(args[n].equals("-h") || args[n].equals("-help")) + { + printArguments(); + return; + } + else if(args[n].equals("-mlgw")) + { + mlgw = true; + } + else if(args[n].equals("-p")) + { + port = Integer.parseInt(args[++n]); + } + else if(args[n].equals("-v") || args[n].equals("-version")) + { + // Simply return as the version info is already printed above + return; + } + } + + // Try to load the JDBCDatabase; + // Do NOT USE BackendConfig or Log classes before this point because they require + // a working JDBCDatabase connection. + try + { + StorageProvider sprov = + StorageManager.loadProvider("org.sonews.storage.impl.JDBCDatabaseProvider"); + StorageManager.enableProvider(sprov); + + // Make sure some elementary groups are existing + if(!StorageManager.current().isGroupExisting("control")) + { + StorageManager.current().addGroup("control", 0); + Log.msg("Group 'control' created.", true); + } + } + catch(StorageBackendException ex) + { + ex.printStackTrace(); + System.err.println("Database initialization failed with " + ex.toString()); + System.err.println("Make sure you have specified the correct database" + + " settings in sonews.conf!"); + return; + } + + ChannelLineBuffers.allocateDirect(); + + // Add shutdown hook + Runtime.getRuntime().addShutdownHook(new ShutdownHook()); + + // Start the listening daemon + if(port <= 0) + { + port = Config.inst().get(Config.PORT, 119); + } + final NNTPDaemon daemon = NNTPDaemon.createInstance(port); + daemon.start(); + + // Start Connections purger thread... + Connections.getInstance().start(); + + // Start mailinglist gateway... + if(mlgw) + { + new MailPoller().start(); + } + + // Start feeds + if(feed) + { + FeedManager.startFeeding(); + } + + Purger purger = new Purger(); + purger.start(); + + // Wait for main thread to exit (setDaemon(false)) + daemon.join(); + } + + private static void printArguments() + { + String usage = Resource.getAsString("helpers/usage", true); + System.out.println(usage); + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/ShutdownHook.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/ShutdownHook.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,84 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews; + +import java.sql.SQLException; +import java.util.Map; +import org.sonews.daemon.AbstractDaemon; + +/** + * Will force all other threads to shutdown cleanly. + * @author Christian Lins + * @since sonews/0.5.0 + */ +class ShutdownHook extends Thread +{ + + /** + * Called when the JVM exits. + */ + @Override + public void run() + { + System.out.println("sonews: Trying to shutdown all threads..."); + + Map threadsMap = Thread.getAllStackTraces(); + for(Thread thread : threadsMap.keySet()) + { + // Interrupt the thread if it's a AbstractDaemon + AbstractDaemon daemon; + if(thread instanceof AbstractDaemon && thread.isAlive()) + { + try + { + daemon = (AbstractDaemon)thread; + daemon.shutdownNow(); + } + catch(SQLException ex) + { + System.out.println("sonews: " + ex); + } + } + } + + for(Thread thread : threadsMap.keySet()) + { + AbstractDaemon daemon; + if(thread instanceof AbstractDaemon && thread.isAlive()) + { + daemon = (AbstractDaemon)thread; + System.out.println("sonews: Waiting for " + daemon + " to exit..."); + try + { + daemon.join(500); + } + catch(InterruptedException ex) + { + System.out.println(ex.getLocalizedMessage()); + } + } + } + + // We have notified all not-sleeping AbstractDaemons of the shutdown; + // all other threads can be simply purged on VM shutdown + + System.out.println("sonews: Clean shutdown."); + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/config/AbstractConfig.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/config/AbstractConfig.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,57 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.config; + +/** + * Base class for Config and BootstrapConfig. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public abstract class AbstractConfig +{ + + public abstract String get(String key, String defVal); + + public int get(final String key, final int defVal) + { + return Integer.parseInt( + get(key, Integer.toString(defVal))); + } + + public boolean get(String key, boolean defVal) + { + String val = get(key, Boolean.toString(defVal)); + return Boolean.parseBoolean(val); + } + + /** + * Returns a long config value specified via the given key. + * @param key + * @param defVal + * @return + */ + public long get(String key, long defVal) + { + String val = get(key, Long.toString(defVal)); + return Long.parseLong(val); + } + + protected abstract void set(String key, String val); + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/config/BackendConfig.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/config/BackendConfig.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,108 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.config; + +import org.sonews.util.Log; +import org.sonews.storage.StorageBackendException; +import org.sonews.storage.StorageManager; +import org.sonews.util.TimeoutMap; + +/** + * Provides access to the program wide configuration that is stored within + * the server's database. + * @author Christian Lins + * @since sonews/0.5.0 + */ +class BackendConfig extends AbstractConfig +{ + + private static BackendConfig instance = new BackendConfig(); + + public static BackendConfig getInstance() + { + return instance; + } + + private final TimeoutMap values + = new TimeoutMap(); + + private BackendConfig() + { + super(); + } + + /** + * Returns the config value for the given key or the defaultValue if the + * key is not found in config. + * @param key + * @param defaultValue + * @return + */ + @Override + public String get(String key, String defaultValue) + { + try + { + String configValue = values.get(key); + if(configValue == null) + { + configValue = StorageManager.current().getConfigValue(key); + if(configValue == null) + { + return defaultValue; + } + else + { + values.put(key, configValue); + return configValue; + } + } + else + { + return configValue; + } + } + catch(StorageBackendException ex) + { + Log.msg(ex.getMessage(), false); + return defaultValue; + } + } + + /** + * Sets the config value which is identified by the given key. + * @param key + * @param value + */ + public void set(String key, String value) + { + values.put(key, value); + + try + { + // Write values to database + StorageManager.current().setConfigValue(key, value); + } + catch(StorageBackendException ex) + { + ex.printStackTrace(); + } + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/config/CommandLineConfig.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/config/CommandLineConfig.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,64 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.config; + +import java.util.Map; +import java.util.HashMap; + +/** + * + * @author Christian Lins + */ +class CommandLineConfig extends AbstractConfig +{ + + private static final CommandLineConfig instance = new CommandLineConfig(); + + public static CommandLineConfig getInstance() + { + return instance; + } + + private final Map values = new HashMap(); + + private CommandLineConfig() {} + + @Override + public String get(String key, String def) + { + synchronized(this.values) + { + if(this.values.containsKey(key)) + { + def = this.values.get(key); + } + } + return def; + } + + @Override + public void set(String key, String val) + { + synchronized(this.values) + { + this.values.put(key, val); + } + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/config/Config.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/config/Config.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,178 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.config; + +/** + * Configuration facade class. + * @author Christian Lins + * @since sonews/1.0 + */ +public class Config extends AbstractConfig +{ + + public static final int LEVEL_CLI = 1; + public static final int LEVEL_FILE = 2; + public static final int LEVEL_BACKEND = 3; + + public static final String CONFIGFILE = "sonews.configfile"; + + /** BackendConfig key constant. Value is the maximum article size in kilobytes. */ + public static final String ARTICLE_MAXSIZE = "sonews.article.maxsize"; + + /** BackendConfig key constant. Value: Amount of news that are feeded per run. */ + public static final String EVENTLOG = "sonews.eventlog"; + public static final String FEED_NEWSPERRUN = "sonews.feed.newsperrun"; + public static final String FEED_PULLINTERVAL = "sonews.feed.pullinterval"; + public static final String HOSTNAME = "sonews.hostname"; + public static final String PORT = "sonews.port"; + public static final String TIMEOUT = "sonews.timeout"; + public static final String MLPOLL_DELETEUNKNOWN = "sonews.mlpoll.deleteunknown"; + public static final String MLPOLL_HOST = "sonews.mlpoll.host"; + public static final String MLPOLL_PASSWORD = "sonews.mlpoll.password"; + public static final String MLPOLL_USER = "sonews.mlpoll.user"; + public static final String MLSEND_ADDRESS = "sonews.mlsend.address"; + public static final String MLSEND_RW_FROM = "sonews.mlsend.rewrite.from"; + public static final String MLSEND_RW_SENDER = "sonews.mlsend.rewrite.sender"; + public static final String MLSEND_HOST = "sonews.mlsend.host"; + public static final String MLSEND_PASSWORD = "sonews.mlsend.password"; + public static final String MLSEND_PORT = "sonews.mlsend.port"; + public static final String MLSEND_USER = "sonews.mlsend.user"; + + /** Key constant. If value is "true" every I/O is written to logfile + * (which is a lot!) + */ + public static final String DEBUG = "sonews.debug"; + + /** Key constant. Value is classname of the JDBC driver */ + public static final String STORAGE_DBMSDRIVER = "sonews.storage.dbmsdriver"; + + /** Key constant. Value is JDBC connect String to the database. */ + public static final String STORAGE_DATABASE = "sonews.storage.database"; + + /** Key constant. Value is the username for the DBMS. */ + public static final String STORAGE_USER = "sonews.storage.user"; + + /** Key constant. Value is the password for the DBMS. */ + public static final String STORAGE_PASSWORD = "sonews.storage.password"; + + /** Key constant. Value is the name of the host which is allowed to use the + * XDAEMON command; default: "localhost" */ + public static final String XDAEMON_HOST = "sonews.xdaemon.host"; + + /** The config key for the filename of the logfile */ + public static final String LOGFILE = "sonews.log"; + + public static final String[] AVAILABLE_KEYS = { + ARTICLE_MAXSIZE, + EVENTLOG, + FEED_NEWSPERRUN, + FEED_PULLINTERVAL, + HOSTNAME, + MLPOLL_DELETEUNKNOWN, + MLPOLL_HOST, + MLPOLL_PASSWORD, + MLPOLL_USER, + MLSEND_ADDRESS, + MLSEND_HOST, + MLSEND_PASSWORD, + MLSEND_PORT, + MLSEND_RW_FROM, + MLSEND_RW_SENDER, + MLSEND_USER, + PORT, + TIMEOUT, + XDAEMON_HOST + }; + + private static Config instance = new Config(); + + public static Config inst() + { + return instance; + } + + private Config(){} + + @Override + public String get(String key, String def) + { + String val = CommandLineConfig.getInstance().get(key, null); + + if(val == null) + { + val = FileConfig.getInstance().get(key, null); + } + + if(val == null) + { + val = BackendConfig.getInstance().get(key, def); + } + + return val; + } + + public String get(int level, String key, String def) + { + switch(level) + { + case LEVEL_CLI: + { + return CommandLineConfig.getInstance().get(key, def); + } + case LEVEL_FILE: + { + return FileConfig.getInstance().get(key, def); + } + case LEVEL_BACKEND: + { + return BackendConfig.getInstance().get(key, def); + } + } + return null; + } + + @Override + public void set(String key, String val) + { + set(LEVEL_BACKEND, key, val); + } + + public void set(int level, String key, String val) + { + switch(level) + { + case LEVEL_CLI: + { + CommandLineConfig.getInstance().set(key, val); + break; + } + case LEVEL_FILE: + { + FileConfig.getInstance().set(key, val); + break; + } + case LEVEL_BACKEND: + { + BackendConfig.getInstance().set(key, val); + break; + } + } + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/config/FileConfig.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/config/FileConfig.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,169 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.config; + +import java.io.FileInputStream; +import java.io.FileNotFoundException; +import java.io.FileOutputStream; +import java.io.IOException; +import java.util.Properties; + +/** + * Manages the bootstrap configuration. It MUST contain all config values + * that are needed to establish a database connection. + * For further configuration values use the Config class instead as that class + * stores its values within the database. + * @author Christian Lins + * @since sonews/0.5.0 + */ +class FileConfig extends AbstractConfig +{ + + private static final Properties defaultConfig = new Properties(); + + private static FileConfig instance = null; + + static + { + // Set some default values + defaultConfig.setProperty(Config.STORAGE_DATABASE, "jdbc:mysql://localhost/sonews"); + defaultConfig.setProperty(Config.STORAGE_DBMSDRIVER, "com.mysql.jdbc.Driver"); + defaultConfig.setProperty(Config.STORAGE_USER, "sonews_user"); + defaultConfig.setProperty(Config.STORAGE_PASSWORD, "mysecret"); + defaultConfig.setProperty(Config.DEBUG, "false"); + } + + /** + * Note: this method is not thread-safe + * @return A Config instance + */ + public static synchronized FileConfig getInstance() + { + if(instance == null) + { + instance = new FileConfig(); + } + return instance; + } + + // Every config instance is initialized with the default values. + private final Properties settings = (Properties)defaultConfig.clone(); + + /** + * Config is a singelton class with only one instance at time. + * So the constructor is private to prevent the creation of more + * then one Config instance. + * @see Config.getInstance() to retrieve an instance of Config + */ + private FileConfig() + { + try + { + // Load settings from file + load(); + } + catch(IOException ex) + { + ex.printStackTrace(); + } + } + + /** + * Loads the configuration from the config file. By default this is done + * by the (private) constructor but it can be useful to reload the config + * by invoking this method. + * @throws IOException + */ + public void load() + throws IOException + { + FileInputStream in = null; + + try + { + in = new FileInputStream( + Config.inst().get(Config.LEVEL_CLI, Config.CONFIGFILE, "sonews.conf")); + settings.load(in); + } + catch (FileNotFoundException e) + { + // MUST NOT use Log otherwise endless loop + System.err.println(e.getMessage()); + save(); + } + finally + { + if(in != null) + in.close(); + } + } + + /** + * Saves this Config to the config file. By default this is done + * at program end. + * @throws FileNotFoundException + * @throws IOException + */ + public void save() throws FileNotFoundException, IOException + { + FileOutputStream out = null; + try + { + out = new FileOutputStream( + Config.inst().get(Config.LEVEL_CLI, Config.CONFIGFILE, "sonews.conf")); + settings.store(out, "SONEWS Config File"); + out.flush(); + } + catch(IOException ex) + { + throw ex; + } + finally + { + if(out != null) + out.close(); + } + } + + /** + * Returns the value that is stored within this config + * identified by the given key. If the key cannot be found + * the default value is returned. + * @param key Key to identify the value. + * @param def The default value that is returned if the key + * is not found in this Config. + * @return + */ + @Override + public String get(String key, String def) + { + return settings.getProperty(key, def); + } + + /** + * Sets the value for a given key. + * @param key + * @param value + */ + public void set(final String key, final String value) + { + settings.setProperty(key, value); + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/AbstractDaemon.java --- a/org/sonews/daemon/AbstractDaemon.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/AbstractDaemon.java Wed Jul 22 14:04:05 2009 +0200 @@ -19,7 +19,7 @@ package org.sonews.daemon; import java.sql.SQLException; -import org.sonews.daemon.storage.Database; +import org.sonews.storage.StorageManager; import org.sonews.util.Log; /** @@ -56,21 +56,17 @@ } /** - * Marks this thread to exit soon. Closes the associated Database connection + * Marks this thread to exit soon. Closes the associated JDBCDatabase connection * if available. * @throws java.sql.SQLException */ - void shutdownNow() + public void shutdownNow() throws SQLException { synchronized(this) { this.isRunning = false; - Database db = Database.getInstance(false); - if(db != null) - { - db.shutdown(); - } + StorageManager.disableProvider(); } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/BootstrapConfig.java --- a/org/sonews/daemon/BootstrapConfig.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,194 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.daemon; - -import java.io.FileInputStream; -import java.io.FileNotFoundException; -import java.io.FileOutputStream; -import java.io.IOException; -import java.util.Properties; -import org.sonews.util.AbstractConfig; - -/** - * Manages the bootstrap configuration. It MUST contain all config values - * that are needed to establish a database connection. - * For further configuration values use the Config class instead as that class - * stores its values within the database. - * @author Christian Lins - * @since sonews/0.5.0 - */ -public final class BootstrapConfig extends AbstractConfig -{ - - /** Key constant. If value is "true" every I/O is written to logfile - * (which is a lot!) - */ - public static final String DEBUG = "sonews.debug"; - - /** Key constant. Value is classname of the JDBC driver */ - public static final String STORAGE_DBMSDRIVER = "sonews.storage.dbmsdriver"; - - /** Key constant. Value is JDBC connect String to the database. */ - public static final String STORAGE_DATABASE = "sonews.storage.database"; - - /** Key constant. Value is the username for the DBMS. */ - public static final String STORAGE_USER = "sonews.storage.user"; - - /** Key constant. Value is the password for the DBMS. */ - public static final String STORAGE_PASSWORD = "sonews.storage.password"; - - /** Key constant. Value is the name of the host which is allowed to use the - * XDAEMON command; default: "localhost" */ - public static final String XDAEMON_HOST = "sonews.xdaemon.host"; - - /** The config key for the filename of the logfile */ - public static final String LOGFILE = "sonews.log"; - - /** The filename of the config file that is loaded on startup */ - public static volatile String FILE = "sonews.conf"; - - private static final Properties defaultConfig = new Properties(); - - private static BootstrapConfig instance = null; - - static - { - // Set some default values - defaultConfig.setProperty(STORAGE_DATABASE, "jdbc:mysql://localhost/sonews"); - defaultConfig.setProperty(STORAGE_DBMSDRIVER, "com.mysql.jdbc.Driver"); - defaultConfig.setProperty(STORAGE_USER, "sonews_user"); - defaultConfig.setProperty(STORAGE_PASSWORD, "mysecret"); - defaultConfig.setProperty(DEBUG, "false"); - } - - /** - * Note: this method is not thread-safe - * @return A Config instance - */ - public static synchronized BootstrapConfig getInstance() - { - if(instance == null) - { - instance = new BootstrapConfig(); - } - return instance; - } - - // Every config instance is initialized with the default values. - private final Properties settings = (Properties)defaultConfig.clone(); - - /** - * Config is a singelton class with only one instance at time. - * So the constructor is private to prevent the creation of more - * then one Config instance. - * @see Config.getInstance() to retrieve an instance of Config - */ - private BootstrapConfig() - { - try - { - // Load settings from file - load(); - } - catch(IOException ex) - { - ex.printStackTrace(); - } - } - - /** - * Loads the configuration from the config file. By default this is done - * by the (private) constructor but it can be useful to reload the config - * by invoking this method. - * @throws IOException - */ - public void load() - throws IOException - { - FileInputStream in = null; - - try - { - in = new FileInputStream(FILE); - settings.load(in); - } - catch (FileNotFoundException e) - { - // MUST NOT use Log otherwise endless loop - System.err.println(e.getMessage()); - save(); - } - finally - { - if(in != null) - in.close(); - } - } - - /** - * Saves this Config to the config file. By default this is done - * at program end. - * @throws FileNotFoundException - * @throws IOException - */ - public void save() throws FileNotFoundException, IOException - { - FileOutputStream out = null; - try - { - out = new FileOutputStream(FILE); - settings.store(out, "SONEWS Config File"); - out.flush(); - } - catch(IOException ex) - { - throw ex; - } - finally - { - if(out != null) - out.close(); - } - } - - /** - * Returns the value that is stored within this config - * identified by the given key. If the key cannot be found - * the default value is returned. - * @param key Key to identify the value. - * @param def The default value that is returned if the key - * is not found in this Config. - * @return - */ - public String get(String key, String def) - { - return settings.getProperty(key, def); - } - - /** - * Sets the value for a given key. - * @param key - * @param value - */ - public void set(final String key, final String value) - { - settings.setProperty(key, value); - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/ChannelLineBuffers.java --- a/org/sonews/daemon/ChannelLineBuffers.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/ChannelLineBuffers.java Wed Jul 22 14:04:05 2009 +0200 @@ -233,10 +233,11 @@ public static void recycleBuffer(ByteBuffer buffer) { assert buffer != null; - assert buffer.capacity() >= BUFFER_SIZE; if(buffer.isDirect()) { + assert buffer.capacity() >= BUFFER_SIZE; + // Add old buffers to the list of free buffers synchronized(freeSmallBuffers) { diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/ChannelReader.java --- a/org/sonews/daemon/ChannelReader.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/ChannelReader.java Wed Jul 22 14:04:05 2009 +0200 @@ -18,7 +18,6 @@ package org.sonews.daemon; -import org.sonews.util.Log; import java.io.IOException; import java.nio.ByteBuffer; import java.nio.channels.CancelledKeyException; @@ -27,6 +26,7 @@ import java.nio.channels.SocketChannel; import java.util.Iterator; import java.util.Set; +import org.sonews.util.Log; /** * A Thread task listening for OP_READ events from SocketChannels. @@ -162,7 +162,7 @@ // Some bytes are available for reading if(selKey.isValid()) - { + { // Lock the channel //synchronized(socketChannel) { @@ -172,7 +172,13 @@ try { read = socketChannel.read(buf); - } + } + catch(IOException ex) + { + // The connection was probably closed by the remote host + // in a non-clean fashion + Log.msg("ChannelReader.processSelectionKey(): " + ex, true); + } catch(Exception ex) { Log.msg("ChannelReader.processSelectionKey(): " + ex, false); diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/CommandSelector.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/CommandSelector.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,110 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon; + +import java.util.HashMap; +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; +import org.sonews.daemon.command.Command; +import org.sonews.daemon.command.UnsupportedCommand; +import org.sonews.util.Log; +import org.sonews.util.io.Resource; + +/** + * Selects the correct command processing class. + * @author Christian Lins + */ +class CommandSelector +{ + + private static Map instances + = new ConcurrentHashMap(); + + public static CommandSelector getInstance() + { + CommandSelector csel = instances.get(Thread.currentThread()); + if(csel == null) + { + csel = new CommandSelector(); + instances.put(Thread.currentThread(), csel); + } + return csel; + } + + private Map commandMapping = new HashMap(); + private Command unsupportedCmd = new UnsupportedCommand(); + + private CommandSelector() + { + String[] classes = Resource.getAsString("helpers/commands.list", true).split("\n"); + for(String className : classes) + { + try + { + Class clazz = Class.forName(className); + Command cmd = (Command)clazz.newInstance(); + String[] cmdStrs = cmd.getSupportedCommandStrings(); + for(String cmdStr : cmdStrs) + { + this.commandMapping.put(cmdStr, cmd); + } + } + catch(ClassNotFoundException ex) + { + Log.msg("Could not load command class: " + ex, false); + } + catch(InstantiationException ex) + { + Log.msg("Could not instantiate command class: " + ex, false); + } + catch(IllegalAccessException ex) + { + Log.msg("Could not access command class: " + ex, false); + } + } + } + + public Command get(String commandName) + { + try + { + commandName = commandName.toUpperCase(); + Command cmd = this.commandMapping.get(commandName); + + if(cmd == null) + { + return this.unsupportedCmd; + } + else if(cmd.isStateful()) + { + return cmd.getClass().newInstance(); + } + else + { + return cmd; + } + } + catch(Exception ex) + { + ex.printStackTrace(); + return this.unsupportedCmd; + } + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/Config.java --- a/org/sonews/daemon/Config.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,149 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.daemon; - -import org.sonews.util.Log; -import java.sql.SQLException; -import org.sonews.daemon.storage.Database; -import org.sonews.util.AbstractConfig; -import org.sonews.util.TimeoutMap; - -/** - * Provides access to the program wide configuration that is stored within - * the server's database. - * @author Christian Lins - * @since sonews/0.5.0 - */ -public final class Config extends AbstractConfig -{ - - /** Config key constant. Value is the maximum article size in kilobytes. */ - public static final String ARTICLE_MAXSIZE = "sonews.article.maxsize"; - - /** Config key constant. Value: Amount of news that are feeded per run. */ - public static final String FEED_NEWSPERRUN = "sonews.feed.newsperrun"; - public static final String FEED_PULLINTERVAL = "sonews.feed.pullinterval"; - public static final String HOSTNAME = "sonews.hostname"; - public static final String PORT = "sonews.port"; - public static final String TIMEOUT = "sonews.timeout"; - public static final String MLPOLL_DELETEUNKNOWN = "sonews.mlpoll.deleteunknown"; - public static final String MLPOLL_HOST = "sonews.mlpoll.host"; - public static final String MLPOLL_PASSWORD = "sonews.mlpoll.password"; - public static final String MLPOLL_USER = "sonews.mlpoll.user"; - public static final String MLSEND_ADDRESS = "sonews.mlsend.address"; - public static final String MLSEND_RW_FROM = "sonews.mlsend.rewrite.from"; - public static final String MLSEND_RW_SENDER = "sonews.mlsend.rewrite.sender"; - public static final String MLSEND_HOST = "sonews.mlsend.host"; - public static final String MLSEND_PASSWORD = "sonews.mlsend.password"; - public static final String MLSEND_PORT = "sonews.mlsend.port"; - public static final String MLSEND_USER = "sonews.mlsend.user"; - - public static final String[] AVAILABLE_KEYS = { - Config.ARTICLE_MAXSIZE, - Config.FEED_NEWSPERRUN, - Config.FEED_PULLINTERVAL, - Config.HOSTNAME, - Config.MLPOLL_DELETEUNKNOWN, - Config.MLPOLL_HOST, - Config.MLPOLL_PASSWORD, - Config.MLPOLL_USER, - Config.MLSEND_ADDRESS, - Config.MLSEND_HOST, - Config.MLSEND_PASSWORD, - Config.MLSEND_PORT, - Config.MLSEND_RW_FROM, - Config.MLSEND_RW_SENDER, - Config.MLSEND_USER, - Config.PORT, - Config.TIMEOUT - }; - - private static Config instance = new Config(); - - public static Config getInstance() - { - return instance; - } - - private final TimeoutMap values - = new TimeoutMap(); - - private Config() - { - super(); - } - - /** - * Returns the config value for the given key or the defaultValue if the - * key is not found in config. - * @param key - * @param defaultValue - * @return - */ - public String get(String key, String defaultValue) - { - try - { - String configValue = values.get(key); - if(configValue == null) - { - configValue = Database.getInstance().getConfigValue(key); - if(configValue == null) - { - return defaultValue; - } - else - { - values.put(key, configValue); - return configValue; - } - } - else - { - return configValue; - } - } - catch(SQLException ex) - { - Log.msg(ex.getMessage(), false); - return defaultValue; - } - } - - /** - * Sets the config value which is identified by the given key. - * @param key - * @param value - */ - public void set(String key, String value) - { - values.put(key, value); - - try - { - // Write values to database - Database.getInstance().setConfigValue(key, value); - } - catch(SQLException ex) - { - ex.printStackTrace(); - } - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/Connections.java --- a/org/sonews/daemon/Connections.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/Connections.java Wed Jul 22 14:04:05 2009 +0200 @@ -18,6 +18,7 @@ package org.sonews.daemon; +import org.sonews.config.Config; import org.sonews.util.Log; import java.io.IOException; import java.net.InetSocketAddress; @@ -37,7 +38,7 @@ * @author Christian Lins * @since sonews/0.5.0 */ -final class Connections extends AbstractDaemon +public final class Connections extends AbstractDaemon { private static final Connections instance = new Connections(); @@ -70,7 +71,7 @@ synchronized(this.connections) { this.connections.add(conn); - this.connByChannel.put(conn.getChannel(), conn); + this.connByChannel.put(conn.getSocketChannel(), conn); } } @@ -95,9 +96,9 @@ for(NNTPConnection conn : this.connections) { assert conn != null; - assert conn.getChannel() != null; + assert conn.getSocketChannel() != null; - Socket socket = conn.getChannel().socket(); + Socket socket = conn.getSocketChannel().socket(); if(socket != null) { InetSocketAddress sockAddr = (InetSocketAddress)socket.getRemoteSocketAddress(); @@ -123,7 +124,7 @@ { while(isRunning()) { - int timeoutMillis = 1000 * Config.getInstance().get(Config.TIMEOUT, 180); + int timeoutMillis = 1000 * Config.inst().get(Config.TIMEOUT, 180); synchronized (this.connections) { @@ -139,7 +140,7 @@ iter.remove(); // Close and remove the channel - SocketChannel channel = conn.getChannel(); + SocketChannel channel = conn.getSocketChannel(); connByChannel.remove(channel); try diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/Main.java --- a/org/sonews/daemon/Main.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,160 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.daemon; - -import java.sql.Driver; -import java.sql.DriverManager; -import java.sql.SQLException; -import java.util.Enumeration; -import java.util.Date; -import org.sonews.feed.FeedManager; -import org.sonews.mlgw.MailPoller; -import org.sonews.daemon.storage.Database; -import org.sonews.util.Log; -import org.sonews.util.io.Resource; - -/** - * Startup class of the daemon. - * @author Christian Lins - * @since sonews/0.5.0 - */ -public final class Main -{ - - private Main() - { - } - - /** Version information of the sonews daemon */ - public static final String VERSION = "sonews/0.6.0beta1"; - public static final Date STARTDATE = new Date(); - - /** - * The main entrypoint. - * @param args - * @throws Exception - */ - public static void main(String[] args) throws Exception - { - System.out.println(VERSION); - Thread.currentThread().setName("Mainthread"); - - // Command line arguments - boolean feed = false; // Enable feeding? - boolean mlgw = false; // Enable Mailinglist gateway? - int port = -1; - - for(int n = 0; n < args.length; n++) - { - if(args[n].equals("-c") || args[n].equals("-config")) - { - BootstrapConfig.FILE = args[++n]; - System.out.println("Using config file " + args[n]); - } - else if(args[n].equals("-dumpjdbcdriver")) - { - System.out.println("Available JDBC drivers:"); - Enumeration drvs = DriverManager.getDrivers(); - while(drvs.hasMoreElements()) - { - System.out.println(drvs.nextElement()); - } - return; - } - else if(args[n].equals("-feed")) - { - feed = true; - } - else if(args[n].equals("-h") || args[n].equals("-help")) - { - printArguments(); - return; - } - else if(args[n].equals("-mlgw")) - { - mlgw = true; - } - else if(args[n].equals("-p")) - { - port = Integer.parseInt(args[++n]); - } - } - - // Try to load the Database; - // Do NOT USE Config or Log classes before this point because they require - // a working Database connection. - try - { - Database.getInstance(); - - // Make sure some elementary groups are existing - if(!Database.getInstance().isGroupExisting("control")) - { - Database.getInstance().addGroup("control", 0); - Log.msg("Group 'control' created.", true); - } - } - catch(SQLException ex) - { - ex.printStackTrace(); - System.err.println("Database initialization failed with " + ex.toString()); - System.err.println("Make sure you have specified the correct database" + - " settings in sonews.conf!"); - return; - } - - ChannelLineBuffers.allocateDirect(); - - // Add shutdown hook - Runtime.getRuntime().addShutdownHook(new ShutdownHook()); - - // Start the listening daemon - if(port <= 0) - { - port = Config.getInstance().get(Config.PORT, 119); - } - final NNTPDaemon daemon = NNTPDaemon.createInstance(port); - daemon.start(); - - // Start Connections purger thread... - Connections.getInstance().start(); - - // Start mailinglist gateway... - if(mlgw) - { - new MailPoller().start(); - } - - // Start feeds - if(feed) - { - FeedManager.startFeeding(); - } - - // Wait for main thread to exit (setDaemon(false)) - daemon.join(); - } - - private static void printArguments() - { - String usage = Resource.getAsString("helpers/usage", true); - System.out.println(usage); - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/NNTPConnection.java --- a/org/sonews/daemon/NNTPConnection.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/NNTPConnection.java Wed Jul 22 14:04:05 2009 +0200 @@ -21,33 +21,19 @@ import org.sonews.util.Log; import java.io.IOException; import java.net.InetSocketAddress; +import java.net.SocketException; import java.nio.ByteBuffer; import java.nio.CharBuffer; import java.nio.channels.ClosedChannelException; import java.nio.channels.SelectionKey; import java.nio.channels.SocketChannel; import java.nio.charset.Charset; +import java.util.Arrays; import java.util.Timer; import java.util.TimerTask; -import org.sonews.daemon.command.ArticleCommand; -import org.sonews.daemon.command.CapabilitiesCommand; -import org.sonews.daemon.command.AbstractCommand; -import org.sonews.daemon.command.GroupCommand; -import org.sonews.daemon.command.HelpCommand; -import org.sonews.daemon.command.ListCommand; -import org.sonews.daemon.command.ListGroupCommand; -import org.sonews.daemon.command.ModeReaderCommand; -import org.sonews.daemon.command.NewGroupsCommand; -import org.sonews.daemon.command.NextPrevCommand; -import org.sonews.daemon.command.OverCommand; -import org.sonews.daemon.command.PostCommand; -import org.sonews.daemon.command.QuitCommand; -import org.sonews.daemon.command.StatCommand; -import org.sonews.daemon.command.UnsupportedCommand; -import org.sonews.daemon.command.XDaemonCommand; -import org.sonews.daemon.command.XPatCommand; -import org.sonews.daemon.storage.Article; -import org.sonews.daemon.storage.Group; +import org.sonews.daemon.command.Command; +import org.sonews.storage.Article; +import org.sonews.storage.Channel; import org.sonews.util.Stats; /** @@ -67,9 +53,9 @@ /** SocketChannel is generally thread-safe */ private SocketChannel channel = null; private Charset charset = Charset.forName("UTF-8"); - private AbstractCommand command = null; + private Command command = null; private Article currentArticle = null; - private Group currentGroup = null; + private Channel currentGroup = null; private volatile long lastActivity = System.currentTimeMillis(); private ChannelLineBuffers lineBuffers = new ChannelLineBuffers(); private int readLock = 0; @@ -201,6 +187,11 @@ channel.socket().shutdownOutput(); channel.close(); } + catch(SocketException ex) + { + // Socket was already disconnected + Log.msg("NNTPConnection.shutdownOutput(): " + ex, true); + } catch(Exception ex) { Log.msg("NNTPConnection.shutdownOutput(): " + ex, false); @@ -213,7 +204,7 @@ }, 3000); } - public SocketChannel getChannel() + public SocketChannel getSocketChannel() { return this.channel; } @@ -227,8 +218,11 @@ { return this.charset; } - - public Group getCurrentGroup() + + /** + * @return The currently selected communication channel (not SocketChannel) + */ + public Channel getCurrentChannel() { return this.currentGroup; } @@ -238,7 +232,7 @@ this.currentArticle = article; } - public void setCurrentGroup(final Group group) + public void setCurrentGroup(final Channel group) { this.currentGroup = group; } @@ -275,6 +269,7 @@ if(line.endsWith("\r")) { line = line.substring(0, line.length() - 1); + raw = Arrays.copyOf(raw, raw.length - 1); } Log.msg("<< " + line, true); @@ -288,7 +283,7 @@ try { // The command object will process the line we just received - command.processLine(line); + command.processLine(this, line, raw); } catch(ClosedChannelException ex0) { @@ -324,86 +319,14 @@ } /** - * This method performes several if/elseif constructs to determine the - * fitting command object. - * TODO: This string comparisons are probably slow! + * This method determines the fitting command processing class. * @param line * @return */ - private AbstractCommand parseCommandLine(String line) + private Command parseCommandLine(String line) { - AbstractCommand cmd = new UnsupportedCommand(this); - String cmdStr = line.split(" ")[0]; - - if(cmdStr.equalsIgnoreCase("ARTICLE") || - cmdStr.equalsIgnoreCase("BODY")) - { - cmd = new ArticleCommand(this); - } - else if(cmdStr.equalsIgnoreCase("CAPABILITIES")) - { - cmd = new CapabilitiesCommand(this); - } - else if(cmdStr.equalsIgnoreCase("GROUP")) - { - cmd = new GroupCommand(this); - } - else if(cmdStr.equalsIgnoreCase("HEAD")) - { - cmd = new ArticleCommand(this); - } - else if(cmdStr.equalsIgnoreCase("HELP")) - { - cmd = new HelpCommand(this); - } - else if(cmdStr.equalsIgnoreCase("LIST")) - { - cmd = new ListCommand(this); - } - else if(cmdStr.equalsIgnoreCase("LISTGROUP")) - { - cmd = new ListGroupCommand(this); - } - else if(cmdStr.equalsIgnoreCase("MODE")) - { - cmd = new ModeReaderCommand(this); - } - else if(cmdStr.equalsIgnoreCase("NEWGROUPS")) - { - cmd = new NewGroupsCommand(this); - } - else if(cmdStr.equalsIgnoreCase("NEXT") || - cmdStr.equalsIgnoreCase("PREV")) - { - cmd = new NextPrevCommand(this); - } - else if(cmdStr.equalsIgnoreCase("OVER") || - cmdStr.equalsIgnoreCase("XOVER")) // for compatibility with older RFCs - { - cmd = new OverCommand(this); - } - else if(cmdStr.equalsIgnoreCase("POST")) - { - cmd = new PostCommand(this); - } - else if(cmdStr.equalsIgnoreCase("QUIT")) - { - cmd = new QuitCommand(this); - } - else if(cmdStr.equalsIgnoreCase("STAT")) - { - cmd = new StatCommand(this); - } - else if(cmdStr.equalsIgnoreCase("XDAEMON")) - { - cmd = new XDaemonCommand(this); - } - else if(cmdStr.equalsIgnoreCase("XPAT")) - { - cmd = new XPatCommand(this); - } - - return cmd; + String cmdStr = line.split(" ")[0]; + return CommandSelector.getInstance().get(cmdStr); } /** @@ -419,6 +342,18 @@ writeToChannel(CharBuffer.wrap(line), charset, line); writeToChannel(CharBuffer.wrap(NEWLINE), charset, null); } + + /** + * Writes the given raw lines to the output buffers and finishes with + * a newline character (\r\n). + * @param rawLines + */ + public void println(final byte[] rawLines) + throws IOException + { + this.lineBuffers.addOutputBuffer(ByteBuffer.wrap(rawLines)); + writeToChannel(CharBuffer.wrap(NEWLINE), charset, null); + } /** * Encodes the given CharBuffer using the given Charset to a bunch of @@ -440,6 +375,11 @@ LineEncoder lenc = new LineEncoder(characters, charset); lenc.encode(lineBuffers); + enableWriteEvents(debugLine); + } + + private void enableWriteEvents(CharSequence debugLine) + { // Enable OP_WRITE events so that the buffers are processed try { diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/NNTPDaemon.java --- a/org/sonews/daemon/NNTPDaemon.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/NNTPDaemon.java Wed Jul 22 14:04:05 2009 +0200 @@ -18,6 +18,8 @@ package org.sonews.daemon; +import org.sonews.config.Config; +import org.sonews.Main; import org.sonews.util.Log; import java.io.IOException; import java.net.BindException; @@ -140,7 +142,7 @@ // Set write selection key and send hello to client conn.setWriteSelectionKey(selKeyWrite); - conn.println("200 " + Config.getInstance().get(Config.HOSTNAME, "localhost") + conn.println("200 " + Config.inst().get(Config.HOSTNAME, "localhost") + " " + Main.VERSION + " news server ready - (posting ok)."); } catch(CancelledKeyException cke) diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/ShutdownHook.java --- a/org/sonews/daemon/ShutdownHook.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,83 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.daemon; - -import java.sql.SQLException; -import java.util.Map; - -/** - * Will force all other threads to shutdown cleanly. - * @author Christian Lins - * @since sonews/0.5.0 - */ -class ShutdownHook extends Thread -{ - - /** - * Called when the JVM exits. - */ - @Override - public void run() - { - System.out.println("sonews: Trying to shutdown all threads..."); - - Map threadsMap = Thread.getAllStackTraces(); - for(Thread thread : threadsMap.keySet()) - { - // Interrupt the thread if it's a AbstractDaemon - AbstractDaemon daemon; - if(thread instanceof AbstractDaemon && thread.isAlive()) - { - try - { - daemon = (AbstractDaemon)thread; - daemon.shutdownNow(); - } - catch(SQLException ex) - { - System.out.println("sonews: " + ex); - } - } - } - - for(Thread thread : threadsMap.keySet()) - { - AbstractDaemon daemon; - if(thread instanceof AbstractDaemon && thread.isAlive()) - { - daemon = (AbstractDaemon)thread; - System.out.println("sonews: Waiting for " + daemon + " to exit..."); - try - { - daemon.join(500); - } - catch(InterruptedException ex) - { - System.out.println(ex.getLocalizedMessage()); - } - } - } - - // We have notified all not-sleeping AbstractDaemons of the shutdown; - // all other threads can be simply purged on VM shutdown - - System.out.println("sonews: Clean shutdown."); - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/AbstractCommand.java --- a/org/sonews/daemon/command/AbstractCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,87 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.daemon.command; - -import java.io.IOException; -import java.nio.charset.Charset; -import java.sql.SQLException; -import org.sonews.daemon.NNTPConnection; -import org.sonews.daemon.storage.Article; -import org.sonews.daemon.storage.Group; - -/** - * Base class for all command handling classes. - * @author Christian Lins - * @author Dennis Schwerdel - * @since n3tpd/0.1 - */ -public abstract class AbstractCommand -{ - - protected NNTPConnection connection; - - public AbstractCommand(final NNTPConnection connection) - { - this.connection = connection; - } - - protected Article getCurrentArticle() - { - return connection.getCurrentArticle(); - } - - protected Group getCurrentGroup() - { - return connection.getCurrentGroup(); - } - - protected void setCurrentArticle(final Article current) - { - connection.setCurrentArticle(current); - } - - protected void setCurrentGroup(final Group current) - { - connection.setCurrentGroup(current); - } - - public abstract void processLine(String line) - throws IOException, SQLException; - - protected void println(final String line) - throws IOException - { - connection.println(line); - } - - protected void println(final String line, final Charset charset) - throws IOException - { - connection.println(line, charset); - } - - protected void printStatus(final int status, final String msg) - throws IOException - { - println(status + " " + msg); - } - - public abstract boolean hasFinished(); - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/ArticleCommand.java --- a/org/sonews/daemon/command/ArticleCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/ArticleCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -19,10 +19,10 @@ package org.sonews.daemon.command; import java.io.IOException; -import java.sql.SQLException; -import org.sonews.daemon.storage.Article; +import org.sonews.storage.Article; import org.sonews.daemon.NNTPConnection; -import org.sonews.daemon.storage.Group; +import org.sonews.storage.Channel; +import org.sonews.storage.StorageBackendException; /** * Class handling the ARTICLE, BODY and HEAD commands. @@ -30,12 +30,13 @@ * @author Dennis Schwerdel * @since n3tpd/0.1 */ -public class ArticleCommand extends AbstractCommand +public class ArticleCommand implements Command { - - public ArticleCommand(final NNTPConnection connection) + + @Override + public String[] getSupportedCommandStrings() { - super(connection); + return new String[] {"ARTICLE", "BODY", "HEAD"}; } @Override @@ -44,9 +45,15 @@ return true; } + @Override + public boolean isStateful() + { + return false; + } + // TODO: Refactor this method to reduce its complexity! @Override - public void processLine(final String line) + public void processLine(NNTPConnection conn, final String line, byte[] raw) throws IOException { final String[] command = line.split(" "); @@ -55,10 +62,10 @@ long artIndex = -1; if (command.length == 1) { - article = getCurrentArticle(); + article = conn.getCurrentArticle(); if (article == null) { - printStatus(420, "no current article has been selected"); + conn.println("420 no current article has been selected"); return; } } @@ -68,7 +75,7 @@ article = Article.getByMessageID(command[1]); if (article == null) { - printStatus(430, "no such article found"); + conn.println("430 no such article found"); return; } } @@ -77,49 +84,47 @@ // Message Number try { - Group currentGroup = connection.getCurrentGroup(); + Channel currentGroup = conn.getCurrentChannel(); if(currentGroup == null) { - printStatus(400, "no group selected"); + conn.println("400 no group selected"); return; } artIndex = Long.parseLong(command[1]); - article = Article.getByArticleNumber(artIndex, currentGroup); + article = currentGroup.getArticle(artIndex); } catch(NumberFormatException ex) { ex.printStackTrace(); } - catch(SQLException ex) + catch(StorageBackendException ex) { ex.printStackTrace(); } if (article == null) { - printStatus(423, "no such article number in this group"); + conn.println("423 no such article number in this group"); return; } - setCurrentArticle(article); + conn.setCurrentArticle(article); } if(command[0].equalsIgnoreCase("ARTICLE")) { - printStatus(220, artIndex + " " + article.getMessageID() + conn.println("220 " + artIndex + " " + article.getMessageID() + " article retrieved - head and body follow"); - - println(article.getHeaderSource()); - - println(""); - println(article.getBody(), article.getBodyCharset()); - println("."); + conn.println(article.getHeaderSource()); + conn.println(""); + conn.println(article.getBody()); + conn.println("."); } else if(command[0].equalsIgnoreCase("BODY")) { - printStatus(222, artIndex + " " + article.getMessageID() + " body"); - println(article.getBody(), article.getBodyCharset()); - println("."); + conn.println("222 " + artIndex + " " + article.getMessageID() + " body"); + conn.println(article.getBody()); + conn.println("."); } /* @@ -153,11 +158,10 @@ */ else if(command[0].equalsIgnoreCase("HEAD")) { - printStatus(221, artIndex + " " + article.getMessageID() + conn.println("221 " + artIndex + " " + article.getMessageID() + " Headers follow (multi-line)"); - - println(article.getHeaderSource()); - println("."); + conn.println(article.getHeaderSource()); + conn.println("."); } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/CapabilitiesCommand.java --- a/org/sonews/daemon/command/CapabilitiesCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/CapabilitiesCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -39,20 +39,21 @@ * @author Christian Lins * @since sonews/0.5.0 */ -public class CapabilitiesCommand extends AbstractCommand +public class CapabilitiesCommand implements Command { - protected static final String[] CAPABILITIES = new String[] + static final String[] CAPABILITIES = new String[] { "VERSION 2", // MUST be the first one; VERSION 2 refers to RFC3977 "READER", // Server implements commands for reading "POST", // Server implements POST command "OVER" // Server implements OVER command }; - - public CapabilitiesCommand(final NNTPConnection conn) + + @Override + public String[] getSupportedCommandStrings() { - super(conn); + return new String[] {"CAPABILITIES"}; } /** @@ -66,15 +67,21 @@ } @Override - public void processLine(final String line) + public boolean isStateful() + { + return false; + } + + @Override + public void processLine(NNTPConnection conn, final String line, byte[] raw) throws IOException { - printStatus(101, "Capabilities list:"); + conn.println("101 Capabilities list:"); for(String cap : CAPABILITIES) { - println(cap); + conn.println(cap); } - println("."); + conn.println("."); } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/Command.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/daemon/command/Command.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,42 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.daemon.command; + +import java.io.IOException; +import org.sonews.daemon.NNTPConnection; +import org.sonews.storage.StorageBackendException; + +/** + * Interface for pluggable NNTP commands handling classes. + * @author Christian Lins + * @since sonews/0.6.0 + */ +public interface Command +{ + + boolean hasFinished(); + + boolean isStateful(); + + String[] getSupportedCommandStrings(); + + void processLine(NNTPConnection conn, String line, byte[] rawLine) + throws IOException, StorageBackendException; + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/GroupCommand.java --- a/org/sonews/daemon/command/GroupCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/GroupCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -19,9 +19,9 @@ package org.sonews.daemon.command; import java.io.IOException; -import java.sql.SQLException; import org.sonews.daemon.NNTPConnection; -import org.sonews.daemon.storage.Group; +import org.sonews.storage.Channel; +import org.sonews.storage.StorageBackendException; /** * Class handling the GROUP command. @@ -45,12 +45,13 @@ * @author Dennis Schwerdel * @since n3tpd/0.1 */ -public class GroupCommand extends AbstractCommand +public class GroupCommand implements Command { - public GroupCommand(final NNTPConnection conn) + @Override + public String[] getSupportedCommandStrings() { - super(conn); + return new String[]{"GROUP"}; } @Override @@ -58,32 +59,37 @@ { return true; } + + @Override + public boolean isStateful() + { + return true; + } @Override - public void processLine(final String line) - throws IOException, SQLException + public void processLine(NNTPConnection conn, final String line, byte[] raw) + throws IOException, StorageBackendException { final String[] command = line.split(" "); - Group group; + Channel group; if(command.length >= 2) { - group = Group.getByName(command[1]); - if(group == null) + group = Channel.getByName(command[1]); + if(group == null || group.isDeleted()) { - printStatus(411, "no such news group"); + conn.println("411 no such news group"); } else { - setCurrentGroup(group); - - printStatus(211, group.getPostingsCount() + " " + group.getFirstArticleNumber() + conn.setCurrentGroup(group); + conn.println("211 " + group.getPostingsCount() + " " + group.getFirstArticleNumber() + " " + group.getLastArticleNumber() + " " + group.getName() + " group selected"); } } else { - printStatus(500, "no group name given"); + conn.println("500 no group name given"); } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/HelpCommand.java --- a/org/sonews/daemon/command/HelpCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/HelpCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -30,34 +30,41 @@ * @author Christian Lins * @since sonews/0.5.0 */ -public class HelpCommand extends AbstractCommand +public class HelpCommand implements Command { - - public HelpCommand(final NNTPConnection conn) - { - super(conn); - } @Override public boolean hasFinished() { return true; } + + @Override + public boolean isStateful() + { + return true; + } + + @Override + public String[] getSupportedCommandStrings() + { + return new String[]{"HELP"}; + } @Override - public void processLine(final String line) + public void processLine(NNTPConnection conn, final String line, byte[] raw) throws IOException { - printStatus(100, "help text follows"); + conn.println("100 help text follows"); final String[] help = Resource .getAsString("helpers/helptext", true).split("\n"); for(String hstr : help) { - println(hstr); + conn.println(hstr); } - println("."); + conn.println("."); } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/ListCommand.java --- a/org/sonews/daemon/command/ListCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/ListCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -19,10 +19,10 @@ package org.sonews.daemon.command; import java.io.IOException; -import java.sql.SQLException; import java.util.List; import org.sonews.daemon.NNTPConnection; -import org.sonews.daemon.storage.Group; +import org.sonews.storage.Channel; +import org.sonews.storage.StorageBackendException; /** * Class handling the LIST command. @@ -30,12 +30,13 @@ * @author Dennis Schwerdel * @since n3tpd/0.1 */ -public class ListCommand extends AbstractCommand +public class ListCommand implements Command { - public ListCommand(final NNTPConnection conn) + @Override + public String[] getSupportedCommandStrings() { - super(conn); + return new String[]{"LIST"}; } @Override @@ -43,10 +44,16 @@ { return true; } + + @Override + public boolean isStateful() + { + return false; + } @Override - public void processLine(final String line) - throws IOException, SQLException + public void processLine(NNTPConnection conn, final String line, byte[] raw) + throws IOException, StorageBackendException { final String[] command = line.split(" "); @@ -54,54 +61,59 @@ { if (command[1].equalsIgnoreCase("OVERVIEW.FMT")) { - printStatus(215, "information follows"); - println("Subject:\nFrom:\nDate:\nMessage-ID:\nReferences:\nBytes:\nLines:\nXref"); - println("."); + conn.println("215 information follows"); + conn.println("Subject:\nFrom:\nDate:\nMessage-ID:\nReferences:\nBytes:\nLines:\nXref"); + conn.println("."); } else if (command[1].equalsIgnoreCase("NEWSGROUPS")) { - printStatus(215, "information follows"); - final List list = Group.getAll(); - for (Group g : list) + conn.println("215 information follows"); + final List list = Channel.getAll(); + for (Channel g : list) { - println(g.getName() + "\t" + "-"); + conn.println(g.getName() + "\t" + "-"); } - println("."); + conn.println("."); } else if (command[1].equalsIgnoreCase("SUBSCRIPTIONS")) { - printStatus(215, "information follows"); - println("."); + conn.println("215 information follows"); + conn.println("."); } else if (command[1].equalsIgnoreCase("EXTENSIONS")) { - printStatus(202, "Supported NNTP extensions."); - println("LISTGROUP"); - println("."); - + conn.println("202 Supported NNTP extensions."); + conn.println("LISTGROUP"); + conn.println("XDAEMON"); + conn.println("XPAT"); + conn.println("."); } else { - printStatus(500, "unknown argument to LIST command"); + conn.println("500 unknown argument to LIST command"); } } else { - final List groups = Group.getAll(); + final List groups = Channel.getAll(); if(groups != null) { - printStatus(215, "list of newsgroups follows"); - for (Group g : groups) + conn.println("215 list of newsgroups follows"); + for (Channel g : groups) { - // Indeed first the higher article number then the lower - println(g.getName() + " " + g.getLastArticleNumber() + " " - + g.getFirstArticleNumber() + " y"); + if(!g.isDeleted()) + { + String writeable = g.isWriteable() ? " y" : " n"; + // Indeed first the higher article number then the lower + conn.println(g.getName() + " " + g.getLastArticleNumber() + " " + + g.getFirstArticleNumber() + writeable); + } } - println("."); + conn.println("."); } else { - printStatus(500, "server database malfunction"); + conn.println("500 server database malfunction"); } } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/ListGroupCommand.java --- a/org/sonews/daemon/command/ListGroupCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/ListGroupCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -19,10 +19,10 @@ package org.sonews.daemon.command; import java.io.IOException; -import java.sql.SQLException; import java.util.List; import org.sonews.daemon.NNTPConnection; -import org.sonews.daemon.storage.Group; +import org.sonews.storage.Channel; +import org.sonews.storage.StorageBackendException; /** * Class handling the LISTGROUP command. @@ -30,12 +30,13 @@ * @author Dennis Schwerdel * @since n3tpd/0.1 */ -public class ListGroupCommand extends AbstractCommand +public class ListGroupCommand implements Command { - public ListGroupCommand(final NNTPConnection conn) + @Override + public String[] getSupportedCommandStrings() { - super(conn); + return new String[]{"LISTGROUP"}; } @Override @@ -45,37 +46,43 @@ } @Override - public void processLine(final String commandName) - throws IOException, SQLException + public boolean isStateful() + { + return false; + } + + @Override + public void processLine(NNTPConnection conn, final String commandName, byte[] raw) + throws IOException, StorageBackendException { final String[] command = commandName.split(" "); - Group group; + Channel group; if(command.length >= 2) { - group = Group.getByName(command[1]); + group = Channel.getByName(command[1]); } else { - group = getCurrentGroup(); + group = conn.getCurrentChannel(); } if (group == null) { - printStatus(412, "no group selected; use GROUP command"); + conn.println("412 no group selected; use GROUP command"); return; } List ids = group.getArticleNumbers(); - printStatus(211, ids.size() + " " + + conn.println("211 " + ids.size() + " " + group.getFirstArticleNumber() + " " + group.getLastArticleNumber() + " list of article numbers follow"); for(long id : ids) { // One index number per line - println(Long.toString(id)); + conn.println(Long.toString(id)); } - println("."); + conn.println("."); } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/ModeReaderCommand.java --- a/org/sonews/daemon/command/ModeReaderCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/ModeReaderCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -19,8 +19,8 @@ package org.sonews.daemon.command; import java.io.IOException; -import java.sql.SQLException; import org.sonews.daemon.NNTPConnection; +import org.sonews.storage.StorageBackendException; /** * Class handling the MODE READER command. This command actually does nothing @@ -28,14 +28,15 @@ * @author Christian Lins * @since sonews/0.5.0 */ -public class ModeReaderCommand extends AbstractCommand +public class ModeReaderCommand implements Command { + + @Override + public String[] getSupportedCommandStrings() + { + return new String[]{"MODE"}; + } - public ModeReaderCommand(final NNTPConnection conn) - { - super(conn); - } - @Override public boolean hasFinished() { @@ -43,15 +44,22 @@ } @Override - public void processLine(final String line) throws IOException, SQLException + public boolean isStateful() + { + return false; + } + + @Override + public void processLine(NNTPConnection conn, final String line, byte[] raw) + throws IOException, StorageBackendException { if(line.equalsIgnoreCase("MODE READER")) { - printStatus(200, "hello you can post"); + conn.println("200 hello you can post"); } else { - printStatus(500, "I do not know this mode command"); + conn.println("500 I do not know this mode command"); } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/NewGroupsCommand.java --- a/org/sonews/daemon/command/NewGroupsCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/NewGroupsCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -19,8 +19,8 @@ package org.sonews.daemon.command; import java.io.IOException; -import java.sql.SQLException; import org.sonews.daemon.NNTPConnection; +import org.sonews.storage.StorageBackendException; /** * Class handling the NEWGROUPS command. @@ -28,12 +28,13 @@ * @author Dennis Schwerdel * @since n3tpd/0.1 */ -public class NewGroupsCommand extends AbstractCommand +public class NewGroupsCommand implements Command { - public NewGroupsCommand(final NNTPConnection conn) + @Override + public String[] getSupportedCommandStrings() { - super(conn); + return new String[]{"NEWGROUPS"}; } @Override @@ -43,22 +44,28 @@ } @Override - public void processLine(final String line) - throws IOException, SQLException + public boolean isStateful() + { + return false; + } + + @Override + public void processLine(NNTPConnection conn, final String line, byte[] raw) + throws IOException, StorageBackendException { final String[] command = line.split(" "); if(command.length == 3) { - printStatus(231, "list of new newsgroups follows"); + conn.println("231 list of new newsgroups follows"); // Currently we do not store a group's creation date; // so we return an empty list which is a valid response - println("."); + conn.println("."); } else { - printStatus(500, "invalid command usage"); + conn.println("500 invalid command usage"); } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/NextPrevCommand.java --- a/org/sonews/daemon/command/NextPrevCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/NextPrevCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -19,10 +19,10 @@ package org.sonews.daemon.command; import java.io.IOException; -import java.sql.SQLException; import org.sonews.daemon.NNTPConnection; -import org.sonews.daemon.storage.Article; -import org.sonews.daemon.storage.Group; +import org.sonews.storage.Article; +import org.sonews.storage.Channel; +import org.sonews.storage.StorageBackendException; /** * Class handling the NEXT and LAST command. @@ -30,12 +30,13 @@ * @author Dennis Schwerdel * @since n3tpd/0.1 */ -public class NextPrevCommand extends AbstractCommand +public class NextPrevCommand implements Command { - public NextPrevCommand(final NNTPConnection conn) + @Override + public String[] getSupportedCommandStrings() { - super(conn); + return new String[]{"NEXT", "PREV"}; } @Override @@ -45,21 +46,27 @@ } @Override - public void processLine(final String line) - throws IOException, SQLException + public boolean isStateful() { - final Article currA = getCurrentArticle(); - final Group currG = getCurrentGroup(); + return false; + } + + @Override + public void processLine(NNTPConnection conn, final String line, byte[] raw) + throws IOException, StorageBackendException + { + final Article currA = conn.getCurrentArticle(); + final Channel currG = conn.getCurrentChannel(); if (currA == null) { - printStatus(420, "no current article has been selected"); + conn.println("420 no current article has been selected"); return; } if (currG == null) { - printStatus(412, "no newsgroup selected"); + conn.println("412 no newsgroup selected"); return; } @@ -67,33 +74,36 @@ if(command[0].equalsIgnoreCase("NEXT")) { - selectNewArticle(currA, currG, 1); + selectNewArticle(conn, currA, currG, 1); } else if(command[0].equalsIgnoreCase("PREV")) { - selectNewArticle(currA, currG, -1); + selectNewArticle(conn, currA, currG, -1); } else { - printStatus(500, "internal server error"); + conn.println("500 internal server error"); } } - private void selectNewArticle(Article article, Group grp, final int delta) - throws IOException, SQLException + private void selectNewArticle(NNTPConnection conn, Article article, Channel grp, + final int delta) + throws IOException, StorageBackendException { assert article != null; - article = Article.getByArticleNumber(article.getIndexInGroup(grp) + delta, grp); + article = grp.getArticle(grp.getIndexOf(article) + delta); if(article == null) { - printStatus(421, "no next article in this group"); + conn.println("421 no next article in this group"); } else { - setCurrentArticle(article); - printStatus(223, article.getIndexInGroup(getCurrentGroup()) + " " + article.getMessageID() + " article retrieved - request text separately"); + conn.setCurrentArticle(article); + conn.println("223 " + conn.getCurrentChannel().getIndexOf(article) + + " " + article.getMessageID() + + " article retrieved - request text separately"); } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/OverCommand.java --- a/org/sonews/daemon/command/OverCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/OverCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -19,13 +19,13 @@ package org.sonews.daemon.command; import java.io.IOException; -import java.sql.SQLException; import java.util.List; import org.sonews.util.Log; import org.sonews.daemon.NNTPConnection; -import org.sonews.daemon.storage.Article; -import org.sonews.daemon.storage.ArticleHead; -import org.sonews.daemon.storage.Headers; +import org.sonews.storage.Article; +import org.sonews.storage.ArticleHead; +import org.sonews.storage.Headers; +import org.sonews.storage.StorageBackendException; import org.sonews.util.Pair; /** @@ -106,14 +106,15 @@ * @author Christian Lins * @since sonews/0.5.0 */ -public class OverCommand extends AbstractCommand +public class OverCommand implements Command { - public static final int MAX_LINES_PER_DBREQUEST = 100; - - public OverCommand(final NNTPConnection conn) + public static final int MAX_LINES_PER_DBREQUEST = 200; + + @Override + public String[] getSupportedCommandStrings() { - super(conn); + return new String[]{"OVER", "XOVER"}; } @Override @@ -123,12 +124,18 @@ } @Override - public void processLine(final String line) - throws IOException, SQLException + public boolean isStateful() { - if(getCurrentGroup() == null) + return false; + } + + @Override + public void processLine(NNTPConnection conn, final String line, byte[] raw) + throws IOException, StorageBackendException + { + if(conn.getCurrentChannel() == null) { - printStatus(412, "No news group current selected"); + conn.println("412 no newsgroup selected"); } else { @@ -138,20 +145,20 @@ // the currently selected article(s) if(command.length == 1) { - final Article art = getCurrentArticle(); + final Article art = conn.getCurrentArticle(); if(art == null) { - printStatus(420, "No article(s) selected"); + conn.println("420 no article(s) selected"); return; } - println(buildOverview(art, -1)); + conn.println(buildOverview(art, -1)); } // otherwise print information about the specified range else { - int artStart; - int artEnd = getCurrentGroup().getLastArticleNumber(); + long artStart; + long artEnd = conn.getCurrentChannel().getLastArticleNumber(); String[] nums = command[1].split("-"); if(nums.length >= 1) { @@ -167,7 +174,7 @@ } else { - artStart = getCurrentGroup().getFirstArticleNumber(); + artStart = conn.getCurrentChannel().getFirstArticleNumber(); } if(nums.length >=2) @@ -186,41 +193,41 @@ { if(command[0].equalsIgnoreCase("OVER")) { - printStatus(423, "No articles in that range"); + conn.println("423 no articles in that range"); } else { - printStatus(224, "(empty) overview information follows:"); - println("."); + conn.println("224 (empty) overview information follows:"); + conn.println("."); } } else { - for(int n = artStart; n <= artEnd; n += MAX_LINES_PER_DBREQUEST) + for(long n = artStart; n <= artEnd; n += MAX_LINES_PER_DBREQUEST) { - int nEnd = Math.min(n + MAX_LINES_PER_DBREQUEST - 1, artEnd); - List> articleHeads = getCurrentGroup() + long nEnd = Math.min(n + MAX_LINES_PER_DBREQUEST - 1, artEnd); + List> articleHeads = conn.getCurrentChannel() .getArticleHeads(n, nEnd); if(articleHeads.isEmpty() && n == artStart && command[0].equalsIgnoreCase("OVER")) { // This reply is only valid for OVER, not for XOVER command - printStatus(423, "No articles in that range"); + conn.println("423 no articles in that range"); return; } else if(n == artStart) { // XOVER replies this although there is no data available - printStatus(224, "Overview information follows"); + conn.println("224 overview information follows"); } for(Pair article : articleHeads) { String overview = buildOverview(article.getB(), article.getA()); - println(overview); + conn.println(overview); } } // for - println("."); + conn.println("."); } } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/PostCommand.java --- a/org/sonews/daemon/command/PostCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/PostCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -19,24 +19,26 @@ package org.sonews.daemon.command; import java.io.IOException; - import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; import java.nio.charset.Charset; import java.nio.charset.IllegalCharsetNameException; import java.nio.charset.UnsupportedCharsetException; import java.sql.SQLException; +import java.util.Arrays; import java.util.Locale; import javax.mail.MessagingException; import javax.mail.internet.AddressException; import javax.mail.internet.InternetHeaders; -import org.sonews.daemon.Config; +import org.sonews.config.Config; import org.sonews.util.Log; import org.sonews.mlgw.Dispatcher; -import org.sonews.daemon.storage.Article; -import org.sonews.daemon.storage.Database; -import org.sonews.daemon.storage.Group; +import org.sonews.storage.Article; +import org.sonews.storage.Group; import org.sonews.daemon.NNTPConnection; -import org.sonews.daemon.storage.Headers; +import org.sonews.storage.Headers; +import org.sonews.storage.StorageBackendException; +import org.sonews.storage.StorageManager; import org.sonews.feed.FeedManager; import org.sonews.util.Stats; @@ -47,7 +49,7 @@ * @author Christian Lins * @since sonews/0.5.0 */ -public class PostCommand extends AbstractCommand +public class PostCommand implements Command { private final Article article = new Article(); @@ -55,14 +57,15 @@ private long bodySize = 0; private InternetHeaders headers = null; private long maxBodySize = - Config.getInstance().get(Config.ARTICLE_MAXSIZE, 128) * 1024L; // Size in bytes + Config.inst().get(Config.ARTICLE_MAXSIZE, 128) * 1024L; // Size in bytes private PostState state = PostState.WaitForLineOne; - private final StringBuilder strBody = new StringBuilder(); - private final StringBuilder strHead = new StringBuilder(); - - public PostCommand(final NNTPConnection conn) + private final ByteArrayOutputStream bufBody = new ByteArrayOutputStream(); + private final StringBuilder strHead = new StringBuilder(); + + @Override + public String[] getSupportedCommandStrings() { - super(conn); + return new String[]{"POST"}; } @Override @@ -71,6 +74,12 @@ return this.state == PostState.Finished; } + @Override + public boolean isStateful() + { + return true; + } + /** * Process the given line String. line.trim() was called by NNTPConnection. * @param line @@ -78,8 +87,8 @@ * @throws java.sql.SQLException */ @Override // TODO: Refactor this method to reduce complexity! - public void processLine(String line) - throws IOException, SQLException + public void processLine(NNTPConnection conn, String line, byte[] raw) + throws IOException, StorageBackendException { switch(state) { @@ -87,12 +96,12 @@ { if(line.equalsIgnoreCase("POST")) { - printStatus(340, "send article to be posted. End with ."); + conn.println("340 send article to be posted. End with ."); state = PostState.ReadingHeaders; } else { - printStatus(500, "invalid command usage"); + conn.println("500 invalid command usage"); } break; } @@ -111,7 +120,7 @@ // Parse the header using the InternetHeader class from JavaMail API headers = new InternetHeaders( new ByteArrayInputStream(strHead.toString().trim() - .getBytes(connection.getCurrentCharset()))); + .getBytes(conn.getCurrentCharset()))); // add the header entries for the article article.setHeaders(headers); @@ -119,21 +128,21 @@ catch (MessagingException e) { e.printStackTrace(); - printStatus(500, "posting failed - invalid header"); + conn.println("500 posting failed - invalid header"); state = PostState.Finished; break; } // Change charset for reading body; // for multipart messages UTF-8 is returned - connection.setCurrentCharset(article.getBodyCharset()); + //conn.setCurrentCharset(article.getBodyCharset()); state = PostState.ReadingBody; if(".".equals(line)) { // Post an article without body - postArticle(article); + postArticle(conn, article); state = PostState.Finished; } } @@ -146,15 +155,16 @@ // Set some headers needed for Over command headers.setHeader(Headers.LINES, Integer.toString(lineCount)); headers.setHeader(Headers.BYTES, Long.toString(bodySize)); + + byte[] body = bufBody.toByteArray(); + if(body.length >= 2) + { + // Remove trailing CRLF + body = Arrays.copyOf(body, body.length - 2); + } + article.setBody(body); // set the article body - if(strBody.length() >= 2) - { - strBody.deleteCharAt(strBody.length() - 1); // Remove last newline - strBody.deleteCharAt(strBody.length() - 1); // Remove last CR - } - article.setBody(strBody.toString()); // set the article body - - postArticle(article); + postArticle(conn, article); state = PostState.Finished; } else @@ -163,19 +173,19 @@ lineCount++; // Add line to body buffer - strBody.append(line); - strBody.append(NNTPConnection.NEWLINE); + bufBody.write(raw, 0, raw.length); + bufBody.write(NNTPConnection.NEWLINE.getBytes()); if(bodySize > maxBodySize) { - printStatus(500, "article is too long"); + conn.println("500 article is too long"); state = PostState.Finished; break; } // Check if this message is a MIME-multipart message and needs a // charset change - try + /*try { line = line.toLowerCase(Locale.ENGLISH); if(line.startsWith(Headers.CONTENT_TYPE)) @@ -197,7 +207,7 @@ try { - connection.setCurrentCharset(Charset.forName(charsetName)); + conn.setCurrentCharset(Charset.forName(charsetName)); } catch(IllegalCharsetNameException ex) { @@ -213,12 +223,15 @@ catch(Exception ex) { ex.printStackTrace(); - } + }*/ } break; } default: + { + // Should never happen Log.msg("PostCommand::processLine(): already finished...", false); + } } } @@ -226,7 +239,7 @@ * Article is a control message and needs special handling. * @param article */ - private void controlMessage(Article article) + private void controlMessage(NNTPConnection conn, Article article) throws IOException { String[] ctrl = article.getHeader(Headers.CONTROL)[0].split(" "); @@ -234,52 +247,52 @@ { try { - Database.getInstance().delete(ctrl[1]); + StorageManager.current().delete(ctrl[1]); // Move cancel message to "control" group article.setHeader(Headers.NEWSGROUPS, "control"); - Database.getInstance().addArticle(article); - printStatus(240, "article cancelled"); + StorageManager.current().addArticle(article); + conn.println("240 article cancelled"); } - catch(SQLException ex) + catch(StorageBackendException ex) { Log.msg(ex, false); - printStatus(500, "internal server error"); + conn.println("500 internal server error"); } } else { - printStatus(441, "unknown Control header"); + conn.println("441 unknown control header"); } } - private void supersedeMessage(Article article) + private void supersedeMessage(NNTPConnection conn, Article article) throws IOException { try { String oldMsg = article.getHeader(Headers.SUPERSEDES)[0]; - Database.getInstance().delete(oldMsg); - Database.getInstance().addArticle(article); - printStatus(240, "article replaced"); + StorageManager.current().delete(oldMsg); + StorageManager.current().addArticle(article); + conn.println("240 article replaced"); } - catch(SQLException ex) + catch(StorageBackendException ex) { Log.msg(ex, false); - printStatus(500, "internal server error"); + conn.println("500 internal server error"); } } - private void postArticle(Article article) + private void postArticle(NNTPConnection conn, Article article) throws IOException { if(article.getHeader(Headers.CONTROL)[0].length() > 0) { - controlMessage(article); + controlMessage(conn, article); } else if(article.getHeader(Headers.SUPERSEDES)[0].length() > 0) { - supersedeMessage(article); + supersedeMessage(conn, article); } else // Post the article regularily { @@ -291,10 +304,10 @@ String[] groupnames = article.getHeader(Headers.NEWSGROUPS)[0].split(","); for(String groupname : groupnames) { - Group group = Database.getInstance().getGroup(groupname); - if(group != null) + Group group = StorageManager.current().getGroup(groupname); + if(group != null && !group.isDeleted()) { - if(group.isMailingList() && !connection.isLocalConnection()) + if(group.isMailingList() && !conn.isLocalConnection()) { // Send to mailing list; the Dispatcher writes // statistics to database @@ -304,9 +317,9 @@ else { // Store in database - if(!Database.getInstance().isArticleExisting(article.getMessageID())) + if(!StorageManager.current().isArticleExisting(article.getMessageID())) { - Database.getInstance().addArticle(article); + StorageManager.current().addArticle(article); // Log this posting to statistics Stats.getInstance().mailPosted( @@ -319,30 +332,30 @@ if(success) { - printStatus(240, "article posted ok"); + conn.println("240 article posted ok"); FeedManager.queueForPush(article); } else { - printStatus(441, "newsgroup not found"); + conn.println("441 newsgroup not found"); } } catch(AddressException ex) { Log.msg(ex.getMessage(), true); - printStatus(441, "invalid sender address"); + conn.println("441 invalid sender address"); } catch(MessagingException ex) { // A MessageException is thrown when the sender email address is // invalid or something is wrong with the SMTP server. System.err.println(ex.getLocalizedMessage()); - printStatus(441, ex.getClass().getCanonicalName() + ": " + ex.getLocalizedMessage()); + conn.println("441 " + ex.getClass().getCanonicalName() + ": " + ex.getLocalizedMessage()); } - catch(SQLException ex) + catch(StorageBackendException ex) { ex.printStackTrace(); - printStatus(500, "internal server error"); + conn.println("500 internal server error"); } } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/QuitCommand.java --- a/org/sonews/daemon/command/QuitCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/QuitCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -19,20 +19,21 @@ package org.sonews.daemon.command; import java.io.IOException; -import java.sql.SQLException; import org.sonews.daemon.NNTPConnection; +import org.sonews.storage.StorageBackendException; /** * Implementation of the QUIT command; client wants to shutdown the connection. * @author Christian Lins * @since sonews/0.5.0 */ -public class QuitCommand extends AbstractCommand +public class QuitCommand implements Command { - public QuitCommand(final NNTPConnection conn) + @Override + public String[] getSupportedCommandStrings() { - super(conn); + return new String[]{"QUIT"}; } @Override @@ -42,13 +43,19 @@ } @Override - public void processLine(final String line) - throws IOException, SQLException + public boolean isStateful() + { + return false; + } + + @Override + public void processLine(NNTPConnection conn, final String line, byte[] raw) + throws IOException, StorageBackendException { - printStatus(205, "cya"); + conn.println("205 cya"); - this.connection.shutdownInput(); - this.connection.shutdownOutput(); + conn.shutdownInput(); + conn.shutdownOutput(); } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/StatCommand.java --- a/org/sonews/daemon/command/StatCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/StatCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -19,21 +19,22 @@ package org.sonews.daemon.command; import java.io.IOException; -import java.sql.SQLException; -import org.sonews.daemon.storage.Article; +import org.sonews.storage.Article; import org.sonews.daemon.NNTPConnection; +import org.sonews.storage.StorageBackendException; /** * Implementation of the STAT command. * @author Christian Lins * @since sonews/0.5.0 */ -public class StatCommand extends AbstractCommand +public class StatCommand implements Command { - public StatCommand(final NNTPConnection conn) + @Override + public String[] getSupportedCommandStrings() { - super(conn); + return new String[]{"STAT"}; } @Override @@ -42,20 +43,26 @@ return true; } + @Override + public boolean isStateful() + { + return false; + } + // TODO: Method has various exit points => Refactor! @Override - public void processLine(final String line) - throws IOException, SQLException + public void processLine(NNTPConnection conn, final String line, byte[] raw) + throws IOException, StorageBackendException { final String[] command = line.split(" "); Article article = null; if(command.length == 1) { - article = getCurrentArticle(); + article = conn.getCurrentArticle(); if(article == null) { - printStatus(420, "no current article has been selected"); + conn.println("420 no current article has been selected"); return; } } @@ -65,7 +72,7 @@ article = Article.getByMessageID(command[1]); if (article == null) { - printStatus(430, "no such article found"); + conn.println("430 no such article found"); return; } } @@ -75,26 +82,27 @@ try { long aid = Long.parseLong(command[1]); - article = Article.getByArticleNumber(aid, getCurrentGroup()); + article = conn.getCurrentChannel().getArticle(aid); } catch(NumberFormatException ex) { ex.printStackTrace(); } - catch(SQLException ex) + catch(StorageBackendException ex) { ex.printStackTrace(); } if (article == null) { - printStatus(423, "no such article number in this group"); + conn.println("423 no such article number in this group"); return; } - setCurrentArticle(article); + conn.setCurrentArticle(article); } - printStatus(223, article.getIndexInGroup(getCurrentGroup()) + " " + article.getMessageID() - + " article retrieved - request text separately"); + conn.println("223 " + conn.getCurrentChannel().getIndexOf(article) + " " + + article.getMessageID() + + " article retrieved - request text separately"); } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/UnsupportedCommand.java --- a/org/sonews/daemon/command/UnsupportedCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/UnsupportedCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -27,14 +27,18 @@ * @author Christian Lins * @since sonews/0.5.0 */ -public class UnsupportedCommand extends AbstractCommand +public class UnsupportedCommand implements Command { + + /** + * @return Always returns null. + */ + @Override + public String[] getSupportedCommandStrings() + { + return null; + } - public UnsupportedCommand(final NNTPConnection conn) - { - super(conn); - } - @Override public boolean hasFinished() { @@ -42,10 +46,16 @@ } @Override - public void processLine(final String line) + public boolean isStateful() + { + return false; + } + + @Override + public void processLine(NNTPConnection conn, final String line, byte[] raw) throws IOException { - printStatus(500, "command not supported"); + conn.println("500 command not supported"); } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/XDaemonCommand.java --- a/org/sonews/daemon/command/XDaemonCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/XDaemonCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -20,15 +20,14 @@ import java.io.IOException; import java.net.InetSocketAddress; -import java.sql.SQLException; import java.util.List; -import org.sonews.daemon.BootstrapConfig; -import org.sonews.daemon.Config; +import org.sonews.config.Config; import org.sonews.daemon.NNTPConnection; -import org.sonews.daemon.storage.Database; -import org.sonews.daemon.storage.Group; +import org.sonews.storage.StorageBackendException; +import org.sonews.storage.StorageManager; import org.sonews.feed.FeedManager; import org.sonews.feed.Subscription; +import org.sonews.storage.Group; import org.sonews.util.Stats; /** @@ -40,12 +39,13 @@ * @author Christian Lins * @since sonews/0.5.0 */ -public class XDaemonCommand extends AbstractCommand +public class XDaemonCommand implements Command { - - public XDaemonCommand(NNTPConnection conn) + + @Override + public String[] getSupportedCommandStrings() { - super(conn); + return new String[]{"XDAEMON"}; } @Override @@ -54,94 +54,102 @@ return true; } + @Override + public boolean isStateful() + { + return false; + } + // TODO: Refactor this method to reduce complexity! @Override - public void processLine(String line) throws IOException, SQLException + public void processLine(NNTPConnection conn, String line, byte[] raw) + throws IOException, StorageBackendException { - InetSocketAddress addr = (InetSocketAddress)connection.getChannel().socket() + InetSocketAddress addr = (InetSocketAddress)conn.getSocketChannel().socket() .getRemoteSocketAddress(); if(addr.getHostName().equals( - BootstrapConfig.getInstance().get(BootstrapConfig.XDAEMON_HOST, "localhost"))) + Config.inst().get(Config.XDAEMON_HOST, "localhost"))) { String[] commands = line.split(" ", 4); if(commands.length == 3 && commands[1].equalsIgnoreCase("LIST")) { if(commands[2].equalsIgnoreCase("CONFIGKEYS")) { - printStatus(200, "list of available config keys follows"); + conn.println("100 list of available config keys follows"); for(String key : Config.AVAILABLE_KEYS) { - println(key); + conn.println(key); } - println("."); + conn.println("."); } else if(commands[2].equalsIgnoreCase("PEERINGRULES")) { List pull = - Database.getInstance().getSubscriptions(FeedManager.TYPE_PULL); + StorageManager.current().getSubscriptions(FeedManager.TYPE_PULL); List push = - Database.getInstance().getSubscriptions(FeedManager.TYPE_PUSH); - printStatus(200,"list of peering rules follows"); + StorageManager.current().getSubscriptions(FeedManager.TYPE_PUSH); + conn.println("100 list of peering rules follows"); for(Subscription sub : pull) { - println("PULL " + sub.getHost() + ":" + sub.getPort() + conn.println("PULL " + sub.getHost() + ":" + sub.getPort() + " " + sub.getGroup()); } for(Subscription sub : push) { - println("PUSH " + sub.getHost() + ":" + sub.getPort() + conn.println("PUSH " + sub.getHost() + ":" + sub.getPort() + " " + sub.getGroup()); } - println("."); + conn.println("."); } else { - printStatus(501, "unknown sub command"); + conn.println("401 unknown sub command"); } } else if(commands.length == 3 && commands[1].equalsIgnoreCase("DELETE")) { - Database.getInstance().delete(commands[2]); - printStatus(200, "article " + commands[2] + " deleted"); + StorageManager.current().delete(commands[2]); + conn.println("200 article " + commands[2] + " deleted"); } else if(commands.length == 4 && commands[1].equalsIgnoreCase("GROUPADD")) { - Database.getInstance().addGroup(commands[2], Integer.parseInt(commands[3])); - printStatus(200, "group " + commands[2] + " created"); + StorageManager.current().addGroup(commands[2], Integer.parseInt(commands[3])); + conn.println("200 group " + commands[2] + " created"); } else if(commands.length == 3 && commands[1].equalsIgnoreCase("GROUPDEL")) { - Group group = Database.getInstance().getGroup(commands[2]); + Group group = StorageManager.current().getGroup(commands[2]); if(group == null) { - printStatus(400, "group not found"); + conn.println("400 group not found"); } else { group.setFlag(Group.DELETED); - printStatus(200, "group " + commands[2] + " marked as deleted"); + group.update(); + conn.println("200 group " + commands[2] + " marked as deleted"); } } else if(commands.length == 4 && commands[1].equalsIgnoreCase("SET")) { String key = commands[2]; String val = commands[3]; - Config.getInstance().set(key, val); - printStatus(200, "new config value set"); + Config.inst().set(key, val); + conn.println("200 new config value set"); } else if(commands.length == 3 && commands[1].equalsIgnoreCase("GET")) { String key = commands[2]; - String val = Config.getInstance().get(key, null); + String val = Config.inst().get(key, null); if(val != null) { - printStatus(200, "config value for " + key + " follows"); - println(val); - println("."); + conn.println("100 config value for " + key + " follows"); + conn.println(val); + conn.println("."); } else { - printStatus(400, "config value not set"); + conn.println("400 config value not set"); } } else if(commands.length >= 3 && commands[1].equalsIgnoreCase("LOG")) @@ -154,83 +162,83 @@ if(commands[2].equalsIgnoreCase("CONNECTED_CLIENTS")) { - printStatus(200, "number of connections follow"); - println(Integer.toString(Stats.getInstance().connectedClients())); - println("."); + conn.println("100 number of connections follow"); + conn.println(Integer.toString(Stats.getInstance().connectedClients())); + conn.println("."); } else if(commands[2].equalsIgnoreCase("POSTED_NEWS")) { - printStatus(200, "hourly numbers of posted news yesterday"); + conn.println("100 hourly numbers of posted news yesterday"); for(int n = 0; n < 24; n++) { - println(n + " " + Stats.getInstance() + conn.println(n + " " + Stats.getInstance() .getYesterdaysEvents(Stats.POSTED_NEWS, n, group)); } - println("."); + conn.println("."); } else if(commands[2].equalsIgnoreCase("GATEWAYED_NEWS")) { - printStatus(200, "hourly numbers of gatewayed news yesterday"); + conn.println("100 hourly numbers of gatewayed news yesterday"); for(int n = 0; n < 24; n++) { - println(n + " " + Stats.getInstance() + conn.println(n + " " + Stats.getInstance() .getYesterdaysEvents(Stats.GATEWAYED_NEWS, n, group)); } - println("."); + conn.println("."); } else if(commands[2].equalsIgnoreCase("TRANSMITTED_NEWS")) { - printStatus(200, "hourly numbers of news transmitted to peers yesterday"); + conn.println("100 hourly numbers of news transmitted to peers yesterday"); for(int n = 0; n < 24; n++) { - println(n + " " + Stats.getInstance() + conn.println(n + " " + Stats.getInstance() .getYesterdaysEvents(Stats.FEEDED_NEWS, n, group)); } - println("."); + conn.println("."); } else if(commands[2].equalsIgnoreCase("HOSTED_NEWS")) { - printStatus(200, "number of overall hosted news"); - println(Integer.toString(Stats.getInstance().getNumberOfNews())); - println("."); + conn.println("100 number of overall hosted news"); + conn.println(Integer.toString(Stats.getInstance().getNumberOfNews())); + conn.println("."); } else if(commands[2].equalsIgnoreCase("HOSTED_GROUPS")) { - printStatus(200, "number of hosted groups"); - println(Integer.toString(Stats.getInstance().getNumberOfGroups())); - println("."); + conn.println("100 number of hosted groups"); + conn.println(Integer.toString(Stats.getInstance().getNumberOfGroups())); + conn.println("."); } else if(commands[2].equalsIgnoreCase("POSTED_NEWS_PER_HOUR")) { - printStatus(200, "posted news per hour"); - println(Double.toString(Stats.getInstance().postedPerHour(-1))); - println("."); + conn.println("100 posted news per hour"); + conn.println(Double.toString(Stats.getInstance().postedPerHour(-1))); + conn.println("."); } else if(commands[2].equalsIgnoreCase("FEEDED_NEWS_PER_HOUR")) { - printStatus(200, "feeded news per hour"); - println(Double.toString(Stats.getInstance().feededPerHour(-1))); - println("."); + conn.println("100 feeded news per hour"); + conn.println(Double.toString(Stats.getInstance().feededPerHour(-1))); + conn.println("."); } else if(commands[2].equalsIgnoreCase("GATEWAYED_NEWS_PER_HOUR")) { - printStatus(200, "gatewayed news per hour"); - println(Double.toString(Stats.getInstance().gatewayedPerHour(-1))); - println("."); + conn.println("100 gatewayed news per hour"); + conn.println(Double.toString(Stats.getInstance().gatewayedPerHour(-1))); + conn.println("."); } else { - printStatus(501, "unknown sub command"); + conn.println("401 unknown sub command"); } } else { - printStatus(500, "invalid command usage"); + conn.println("400 invalid command usage"); } } else { - printStatus(500, "not allowed"); + conn.println("501 not allowed"); } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/command/XPatCommand.java --- a/org/sonews/daemon/command/XPatCommand.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/daemon/command/XPatCommand.java Wed Jul 22 14:04:05 2009 +0200 @@ -19,8 +19,13 @@ package org.sonews.daemon.command; import java.io.IOException; -import java.sql.SQLException; +import java.util.List; +import java.util.Locale; +import java.util.regex.PatternSyntaxException; import org.sonews.daemon.NNTPConnection; +import org.sonews.storage.StorageBackendException; +import org.sonews.storage.StorageManager; +import org.sonews.util.Pair; /** *
@@ -59,18 +64,24 @@
  *       221 Header follows
  *       430 no such article
  *       502 no permission
+ *
+ *   Response Data:
+ *
+ *       art_nr fitting_header_value
+ * 
  * 
* [Source:"draft-ietf-nntp-imp-02.txt"] [Copyright: 1998 S. Barber] * * @author Christian Lins * @since sonews/0.5.0 */ -public class XPatCommand extends AbstractCommand +public class XPatCommand implements Command { - public XPatCommand(final NNTPConnection conn) + @Override + public String[] getSupportedCommandStrings() { - super(conn); + return new String[]{"XPAT"}; } @Override @@ -80,10 +91,74 @@ } @Override - public void processLine(final String line) - throws IOException, SQLException + public boolean isStateful() { - printStatus(500, "not (yet) supported"); + return false; + } + + @Override + public void processLine(NNTPConnection conn, final String line, byte[] raw) + throws IOException, StorageBackendException + { + if(conn.getCurrentChannel() == null) + { + conn.println("430 no group selected"); + return; + } + + String[] command = line.split("\\p{Space}+"); + + // There may be multiple patterns and Thunderbird produces + // additional spaces between range and pattern + if(command.length >= 4) + { + String header = command[1].toLowerCase(Locale.US); + String range = command[2]; + String pattern = command[3]; + + long start = -1; + long end = -1; + if(range.contains("-")) + { + String[] rsplit = range.split("-", 2); + start = Long.parseLong(rsplit[0]); + if(rsplit[1].length() > 0) + { + end = Long.parseLong(rsplit[1]); + } + } + else // TODO: Handle Message-IDs + { + start = Long.parseLong(range); + } + + try + { + List> heads = StorageManager.current(). + getArticleHeaders(conn.getCurrentChannel(), start, end, header, pattern); + + conn.println("221 header follows"); + for(Pair head : heads) + { + conn.println(head.getA() + " " + head.getB()); + } + conn.println("."); + } + catch(PatternSyntaxException ex) + { + ex.printStackTrace(); + conn.println("500 invalid pattern syntax"); + } + catch(StorageBackendException ex) + { + ex.printStackTrace(); + conn.println("500 internal server error"); + } + } + else + { + conn.println("430 invalid command usage"); + } } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/storage/Article.java --- a/org/sonews/daemon/storage/Article.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,401 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.daemon.storage; - -import org.sonews.daemon.Config; -import java.io.BufferedReader; -import java.io.ByteArrayInputStream; -import java.io.IOException; -import java.io.InputStream; -import java.io.InputStreamReader; -import java.nio.charset.Charset; -import java.sql.SQLException; -import java.util.UUID; -import java.util.ArrayList; -import java.util.Enumeration; -import java.util.List; -import javax.mail.Header; -import javax.mail.Message; -import javax.mail.MessagingException; -import javax.mail.Multipart; -import javax.mail.internet.InternetHeaders; -import javax.mail.internet.MimeUtility; -import org.sonews.util.Log; - -/** - * Represents a newsgroup article. - * @author Christian Lins - * @author Denis Schwerdel - * @since n3tpd/0.1 - */ -public class Article extends ArticleHead -{ - - /** - * Loads the Article identified by the given ID from the Database. - * @param messageID - * @return null if Article is not found or if an error occurred. - */ - public static Article getByMessageID(final String messageID) - { - try - { - return Database.getInstance().getArticle(messageID); - } - catch(SQLException ex) - { - ex.printStackTrace(); - return null; - } - } - - public static Article getByArticleNumber(long articleIndex, Group group) - throws SQLException - { - return Database.getInstance().getArticle(articleIndex, group.getID()); - } - - private String body = ""; - private String headerSrc = null; - - /** - * Default constructor. - */ - public Article() - { - } - - /** - * Creates a new Article object using the date from the given - * raw data. - * This construction has only package visibility. - */ - Article(String headers, String body) - { - try - { - this.body = body; - - // Parse the header - this.headers = new InternetHeaders( - new ByteArrayInputStream(headers.getBytes())); - - this.headerSrc = headers; - } - catch(MessagingException ex) - { - ex.printStackTrace(); - } - } - - /** - * Creates an Article instance using the data from the javax.mail.Message - * object. - * @see javax.mail.Message - * @param msg - * @throws IOException - * @throws MessagingException - */ - public Article(final Message msg) - throws IOException, MessagingException - { - this.headers = new InternetHeaders(); - - for(Enumeration e = msg.getAllHeaders() ; e.hasMoreElements();) - { - final Header header = (Header)e.nextElement(); - this.headers.addHeader(header.getName(), header.getValue()); - } - - // The "content" of the message can be a String if it's a simple text/plain - // message, a Multipart object or an InputStream if the content is unknown. - final Object content = msg.getContent(); - if(content instanceof String) - { - this.body = (String)content; - } - else if(content instanceof Multipart) // probably subclass MimeMultipart - { - // We're are not interested in the different parts of the MultipartMessage, - // so we simply read in all data which *can* be huge. - InputStream in = msg.getInputStream(); - this.body = readContent(in); - } - else if(content instanceof InputStream) - { - // The message format is unknown to the Message class, but we can - // simply read in the whole message data. - this.body = readContent((InputStream)content); - } - else - { - // Unknown content is probably a malformed mail we should skip. - // On the other hand we produce an inconsistent mail mirror, but no - // mail system must transport invalid content. - Log.msg("Skipping message due to unknown content. Throwing exception...", true); - throw new MessagingException("Unknown content: " + content); - } - - // Validate headers - validateHeaders(); - } - - /** - * Reads lines from the given InputString into a String object. - * TODO: Move this generalized method to org.sonews.util.io.Resource. - * @param in - * @return - * @throws IOException - */ - private String readContent(InputStream in) - throws IOException - { - StringBuilder buf = new StringBuilder(); - - BufferedReader rin = new BufferedReader(new InputStreamReader(in)); - String line = rin.readLine(); - while(line != null) - { - buf.append('\n'); - buf.append(line); - line = rin.readLine(); - } - - return buf.toString(); - } - - /** - * Removes the header identified by the given key. - * @param headerKey - */ - public void removeHeader(final String headerKey) - { - this.headers.removeHeader(headerKey); - this.headerSrc = null; - } - - /** - * Generates a message id for this article and sets it into - * the header object. You have to update the Database manually to make this - * change persistent. - * Note: a Message-ID should never be changed and only generated once. - */ - private String generateMessageID() - { - String msgID = "<" + UUID.randomUUID() + "@" - + Config.getInstance().get(Config.HOSTNAME, "localhost") + ">"; - - this.headers.setHeader(Headers.MESSAGE_ID, msgID); - - return msgID; - } - - /** - * Returns the body string. - */ - public String getBody() - { - return body; - } - - /** - * @return Charset of the body text - */ - public Charset getBodyCharset() - { - // We espect something like - // Content-Type: text/plain; charset=ISO-8859-15 - String contentType = getHeader(Headers.CONTENT_TYPE)[0]; - int idxCharsetStart = contentType.indexOf("charset=") + "charset=".length(); - int idxCharsetEnd = contentType.indexOf(";", idxCharsetStart); - - String charsetName = "UTF-8"; - if(idxCharsetStart >= 0 && idxCharsetStart < contentType.length()) - { - if(idxCharsetEnd < 0) - { - charsetName = contentType.substring(idxCharsetStart); - } - else - { - charsetName = contentType.substring(idxCharsetStart, idxCharsetEnd); - } - } - - // Sometimes there are '"' around the name - if(charsetName.length() > 2 && - charsetName.charAt(0) == '"' && charsetName.endsWith("\"")) - { - charsetName = charsetName.substring(1, charsetName.length() - 2); - } - - // Create charset - Charset charset = Charset.forName("UTF-8"); // This MUST be supported by JVM - try - { - charset = Charset.forName(charsetName); - } - catch(Exception ex) - { - Log.msg(ex.getMessage(), false); - Log.msg("Article.getBodyCharset(): Unknown charset: " + charsetName, false); - } - return charset; - } - - /** - * @return Numerical IDs of the newsgroups this Article belongs to. - */ - List getGroups() - { - String[] groupnames = getHeader(Headers.NEWSGROUPS)[0].split(","); - ArrayList groups = new ArrayList(); - - try - { - for(String newsgroup : groupnames) - { - newsgroup = newsgroup.trim(); - Group group = Database.getInstance().getGroup(newsgroup); - if(group != null && // If the server does not provide the group, ignore it - !groups.contains(group)) // Yes, there may be duplicates - { - groups.add(group); - } - } - } - catch (SQLException ex) - { - ex.printStackTrace(); - return null; - } - return groups; - } - - public void setBody(String body) - { - this.body = body; - } - - /** - * - * @param groupname Name(s) of newsgroups - */ - public void setGroup(String groupname) - { - this.headers.setHeader(Headers.NEWSGROUPS, groupname); - } - - public String getMessageID() - { - String[] msgID = getHeader(Headers.MESSAGE_ID); - return msgID[0]; - } - - public Enumeration getAllHeaders() - { - return this.headers.getAllHeaders(); - } - - /** - * @return Header source code of this Article. - */ - public String getHeaderSource() - { - if(this.headerSrc != null) - { - return this.headerSrc; - } - - StringBuffer buf = new StringBuffer(); - - for(Enumeration en = this.headers.getAllHeaders(); en.hasMoreElements();) - { - Header entry = (Header)en.nextElement(); - - buf.append(entry.getName()); - buf.append(": "); - buf.append( - MimeUtility.fold(entry.getName().length() + 2, entry.getValue())); - - if(en.hasMoreElements()) - { - buf.append("\r\n"); - } - } - - this.headerSrc = buf.toString(); - return this.headerSrc; - } - - public long getIndexInGroup(Group group) - throws SQLException - { - return Database.getInstance().getArticleIndex(this, group); - } - - /** - * Sets the headers of this Article. If headers contain no - * Message-Id a new one is created. - * @param headers - */ - public void setHeaders(InternetHeaders headers) - { - this.headers = headers; - validateHeaders(); - } - - /** - * @return String containing the Message-ID. - */ - @Override - public String toString() - { - return getMessageID(); - } - - /** - * Checks some headers for their validity and generates an - * appropriate Path-header for this host if not yet existing. - * This method is called by some Article constructors and the - * method setHeaders(). - * @return true if something on the headers was changed. - */ - private void validateHeaders() - { - // Check for valid Path-header - final String path = getHeader(Headers.PATH)[0]; - final String host = Config.getInstance().get(Config.HOSTNAME, "localhost"); - if(!path.startsWith(host)) - { - StringBuffer pathBuf = new StringBuffer(); - pathBuf.append(host); - pathBuf.append('!'); - pathBuf.append(path); - this.headers.setHeader(Headers.PATH, pathBuf.toString()); - } - - // Generate a messageID if no one is existing - if(getMessageID().equals("")) - { - generateMessageID(); - } - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/storage/ArticleHead.java --- a/org/sonews/daemon/storage/ArticleHead.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,78 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.daemon.storage; - -import java.io.ByteArrayInputStream; -import javax.mail.MessagingException; -import javax.mail.internet.InternetHeaders; - -/** - * An article with no body only headers. - * @author Christian Lins - * @since sonews/0.5.0 - */ -public class ArticleHead -{ - - protected InternetHeaders headers; - - protected ArticleHead() - { - } - - public ArticleHead(String headers) - { - try - { - // Parse the header - this.headers = new InternetHeaders( - new ByteArrayInputStream(headers.getBytes())); - } - catch(MessagingException ex) - { - ex.printStackTrace(); - } - } - - /** - * Returns the header field with given name. - * @param name - * @return Header values or empty string. - */ - public String[] getHeader(String name) - { - String[] ret = this.headers.getHeader(name); - if(ret == null) - { - ret = new String[]{""}; - } - return ret; - } - - /** - * Sets the header value identified through the header name. - * @param name - * @param value - */ - public void setHeader(String name, String value) - { - this.headers.setHeader(name, value); - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/storage/Database.java --- a/org/sonews/daemon/storage/Database.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1352 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.daemon.storage; - -import java.sql.Connection; -import java.sql.DriverManager; -import java.sql.ResultSet; -import java.sql.SQLException; -import java.sql.Statement; -import java.sql.PreparedStatement; -import java.util.ArrayList; -import java.util.Enumeration; -import java.util.List; -import java.util.Map; -import java.util.concurrent.ConcurrentHashMap; -import javax.mail.Header; -import javax.mail.internet.InternetAddress; -import javax.mail.internet.MimeUtility; -import org.sonews.daemon.BootstrapConfig; -import org.sonews.util.Log; -import org.sonews.feed.Subscription; -import org.sonews.util.Pair; - -/** - * Database facade class. - * @author Christian Lins - * @since sonews/0.5.0 - */ -// TODO: Refactor this class to reduce size (e.g. ArticleDatabase GroupDatabase) -public class Database -{ - - public static final int MAX_RESTARTS = 3; - - private static final Map instances - = new ConcurrentHashMap(); - - /** - * @return Instance of the current Database backend. Returns null if an error - * has occurred. - */ - public static Database getInstance(boolean create) - throws SQLException - { - if(!instances.containsKey(Thread.currentThread()) && create) - { - Database db = new Database(); - db.arise(); - instances.put(Thread.currentThread(), db); - return db; - } - else - { - return instances.get(Thread.currentThread()); - } - } - - public static Database getInstance() - throws SQLException - { - return getInstance(true); - } - - private Connection conn = null; - private PreparedStatement pstmtAddArticle1 = null; - private PreparedStatement pstmtAddArticle2 = null; - private PreparedStatement pstmtAddArticle3 = null; - private PreparedStatement pstmtAddArticle4 = null; - private PreparedStatement pstmtAddGroup0 = null; - private PreparedStatement pstmtAddEvent = null; - private PreparedStatement pstmtCountArticles = null; - private PreparedStatement pstmtCountGroups = null; - private PreparedStatement pstmtDeleteArticle0 = null; - private PreparedStatement pstmtGetArticle0 = null; - private PreparedStatement pstmtGetArticle1 = null; - private PreparedStatement pstmtGetArticleHeaders = null; - private PreparedStatement pstmtGetArticleHeads = null; - private PreparedStatement pstmtGetArticleIDs = null; - private PreparedStatement pstmtGetArticleIndex = null; - private PreparedStatement pstmtGetConfigValue = null; - private PreparedStatement pstmtGetEventsCount0 = null; - private PreparedStatement pstmtGetEventsCount1 = null; - private PreparedStatement pstmtGetGroupForList = null; - private PreparedStatement pstmtGetGroup0 = null; - private PreparedStatement pstmtGetGroup1 = null; - private PreparedStatement pstmtGetFirstArticleNumber = null; - private PreparedStatement pstmtGetListForGroup = null; - private PreparedStatement pstmtGetLastArticleNumber = null; - private PreparedStatement pstmtGetMaxArticleID = null; - private PreparedStatement pstmtGetMaxArticleIndex = null; - private PreparedStatement pstmtGetPostingsCount = null; - private PreparedStatement pstmtGetSubscriptions = null; - private PreparedStatement pstmtIsArticleExisting = null; - private PreparedStatement pstmtIsGroupExisting = null; - private PreparedStatement pstmtSetConfigValue0 = null; - private PreparedStatement pstmtSetConfigValue1 = null; - - /** How many times the database connection was reinitialized */ - private int restarts = 0; - - /** - * Rises the database: reconnect and recreate all prepared statements. - * @throws java.lang.SQLException - */ - private void arise() - throws SQLException - { - try - { - // Load database driver - Class.forName( - BootstrapConfig.getInstance().get(BootstrapConfig.STORAGE_DBMSDRIVER, "java.lang.Object")); - - // Establish database connection - this.conn = DriverManager.getConnection( - BootstrapConfig.getInstance().get(BootstrapConfig.STORAGE_DATABASE, ""), - BootstrapConfig.getInstance().get(BootstrapConfig.STORAGE_USER, "root"), - BootstrapConfig.getInstance().get(BootstrapConfig.STORAGE_PASSWORD, "")); - - this.conn.setTransactionIsolation(Connection.TRANSACTION_SERIALIZABLE); - if(this.conn.getTransactionIsolation() != Connection.TRANSACTION_SERIALIZABLE) - { - Log.msg("Warning: Database is NOT fully serializable!", false); - } - - // Prepare statements for method addArticle() - this.pstmtAddArticle1 = conn.prepareStatement( - "INSERT INTO articles (article_id, body) VALUES(?, ?)"); - this.pstmtAddArticle2 = conn.prepareStatement( - "INSERT INTO headers (article_id, header_key, header_value, header_index) " + - "VALUES (?, ?, ?, ?)"); - this.pstmtAddArticle3 = conn.prepareStatement( - "INSERT INTO postings (group_id, article_id, article_index)" + - "VALUES (?, ?, ?)"); - this.pstmtAddArticle4 = conn.prepareStatement( - "INSERT INTO article_ids (article_id, message_id) VALUES (?, ?)"); - - // Prepare statement for method addStatValue() - this.pstmtAddEvent = conn.prepareStatement( - "INSERT INTO events VALUES (?, ?, ?)"); - - // Prepare statement for method addGroup() - this.pstmtAddGroup0 = conn.prepareStatement( - "INSERT INTO groups (name, flags) VALUES (?, ?)"); - - // Prepare statement for method countArticles() - this.pstmtCountArticles = conn.prepareStatement( - "SELECT Count(article_id) FROM article_ids"); - - // Prepare statement for method countGroups() - this.pstmtCountGroups = conn.prepareStatement( - "SELECT Count(group_id) FROM groups WHERE " + - "flags & " + Group.DELETED + " = 0"); - - // Prepare statements for method delete(article) - this.pstmtDeleteArticle0 = conn.prepareStatement( - "DELETE FROM articles WHERE article_id = " + - "(SELECT article_id FROM article_ids WHERE message_id = ?)"); - - // Prepare statements for methods getArticle() - this.pstmtGetArticle0 = conn.prepareStatement( - "SELECT * FROM articles WHERE article_id = " + - "(SELECT article_id FROM article_ids WHERE message_id = ?)"); - this.pstmtGetArticle1 = conn.prepareStatement( - "SELECT * FROM articles WHERE article_id = " + - "(SELECT article_id FROM postings WHERE " + - "article_index = ? AND group_id = ?)"); - - // Prepare statement for method getArticleHeaders() - this.pstmtGetArticleHeaders = conn.prepareStatement( - "SELECT header_key, header_value FROM headers WHERE article_id = ? " + - "ORDER BY header_index ASC"); - - this.pstmtGetArticleIDs = conn.prepareStatement( - "SELECT article_index FROM postings WHERE group_id = ?"); - - // Prepare statement for method getArticleIndex - this.pstmtGetArticleIndex = conn.prepareStatement( - "SELECT article_index FROM postings WHERE " + - "article_id = (SELECT article_id FROM article_ids " + - "WHERE message_id = ?) " + - " AND group_id = ?"); - - // Prepare statements for method getArticleHeads() - this.pstmtGetArticleHeads = conn.prepareStatement( - "SELECT article_id, article_index FROM postings WHERE " + - "postings.group_id = ? AND article_index >= ? AND " + - "article_index <= ?"); - - // Prepare statements for method getConfigValue() - this.pstmtGetConfigValue = conn.prepareStatement( - "SELECT config_value FROM config WHERE config_key = ?"); - - // Prepare statements for method getEventsCount() - this.pstmtGetEventsCount0 = conn.prepareStatement( - "SELECT Count(*) FROM events WHERE event_key = ? AND " + - "event_time >= ? AND event_time < ?"); - - this.pstmtGetEventsCount1 = conn.prepareStatement( - "SELECT Count(*) FROM events WHERE event_key = ? AND " + - "event_time >= ? AND event_time < ? AND group_id = ?"); - - // Prepare statement for method getGroupForList() - this.pstmtGetGroupForList = conn.prepareStatement( - "SELECT name FROM groups INNER JOIN groups2list " + - "ON groups.group_id = groups2list.group_id " + - "WHERE groups2list.listaddress = ?"); - - // Prepare statement for method getGroup() - this.pstmtGetGroup0 = conn.prepareStatement( - "SELECT group_id, flags FROM groups WHERE Name = ?"); - this.pstmtGetGroup1 = conn.prepareStatement( - "SELECT name FROM groups WHERE group_id = ?"); - - // Prepare statement for method getLastArticleNumber() - this.pstmtGetLastArticleNumber = conn.prepareStatement( - "SELECT Max(article_index) FROM postings WHERE group_id = ?"); - - // Prepare statement for method getListForGroup() - this.pstmtGetListForGroup = conn.prepareStatement( - "SELECT listaddress FROM groups2list INNER JOIN groups " + - "ON groups.group_id = groups2list.group_id WHERE name = ?"); - - // Prepare statement for method getMaxArticleID() - this.pstmtGetMaxArticleID = conn.prepareStatement( - "SELECT Max(article_id) FROM articles"); - - // Prepare statement for method getMaxArticleIndex() - this.pstmtGetMaxArticleIndex = conn.prepareStatement( - "SELECT Max(article_index) FROM postings WHERE group_id = ?"); - - // Prepare statement for method getFirstArticleNumber() - this.pstmtGetFirstArticleNumber = conn.prepareStatement( - "SELECT Min(article_index) FROM postings WHERE group_id = ?"); - - // Prepare statement for method getPostingsCount() - this.pstmtGetPostingsCount = conn.prepareStatement( - "SELECT Count(*) FROM postings NATURAL JOIN groups " + - "WHERE groups.name = ?"); - - // Prepare statement for method getSubscriptions() - this.pstmtGetSubscriptions = conn.prepareStatement( - "SELECT host, port, name FROM peers NATURAL JOIN " + - "peer_subscriptions NATURAL JOIN groups WHERE feedtype = ?"); - - // Prepare statement for method isArticleExisting() - this.pstmtIsArticleExisting = conn.prepareStatement( - "SELECT Count(article_id) FROM article_ids WHERE message_id = ?"); - - // Prepare statement for method isGroupExisting() - this.pstmtIsGroupExisting = conn.prepareStatement( - "SELECT * FROM groups WHERE name = ?"); - - // Prepare statement for method setConfigValue() - this.pstmtSetConfigValue0 = conn.prepareStatement( - "DELETE FROM config WHERE config_key = ?"); - this.pstmtSetConfigValue1 = conn.prepareStatement( - "INSERT INTO config VALUES(?, ?)"); - } - catch(ClassNotFoundException ex) - { - throw new Error("JDBC Driver not found!", ex); - } - } - - /** - * Adds an article to the database. - * @param article - * @return - * @throws java.sql.SQLException - */ - public void addArticle(final Article article) - throws SQLException - { - try - { - this.conn.setAutoCommit(false); - - int newArticleID = getMaxArticleID() + 1; - - // Fill prepared statement with values; - // writes body to article table - pstmtAddArticle1.setInt(1, newArticleID); - pstmtAddArticle1.setBytes(2, article.getBody().getBytes()); - pstmtAddArticle1.execute(); - - // Add headers - Enumeration headers = article.getAllHeaders(); - for(int n = 0; headers.hasMoreElements(); n++) - { - Header header = (Header)headers.nextElement(); - pstmtAddArticle2.setInt(1, newArticleID); - pstmtAddArticle2.setString(2, header.getName().toLowerCase()); - pstmtAddArticle2.setString(3, - header.getValue().replaceAll("[\r\n]", "")); - pstmtAddArticle2.setInt(4, n); - pstmtAddArticle2.execute(); - } - - // For each newsgroup add a reference - List groups = article.getGroups(); - for(Group group : groups) - { - pstmtAddArticle3.setLong(1, group.getID()); - pstmtAddArticle3.setInt(2, newArticleID); - pstmtAddArticle3.setLong(3, getMaxArticleIndex(group.getID()) + 1); - pstmtAddArticle3.execute(); - } - - // Write message-id to article_ids table - this.pstmtAddArticle4.setInt(1, newArticleID); - this.pstmtAddArticle4.setString(2, article.getMessageID()); - this.pstmtAddArticle4.execute(); - - this.conn.commit(); - this.conn.setAutoCommit(true); - - this.restarts = 0; // Reset error count - } - catch(SQLException ex) - { - try - { - this.conn.rollback(); // Rollback changes - } - catch(SQLException ex2) - { - Log.msg("Rollback of addArticle() failed: " + ex2, false); - } - - try - { - this.conn.setAutoCommit(true); // and release locks - } - catch(SQLException ex2) - { - Log.msg("setAutoCommit(true) of addArticle() failed: " + ex2, false); - } - - restartConnection(ex); - addArticle(article); - } - } - - /** - * Adds a group to the Database. This method is not accessible via NNTP. - * @param name - * @throws java.sql.SQLException - */ - public void addGroup(String name, int flags) - throws SQLException - { - try - { - this.conn.setAutoCommit(false); - pstmtAddGroup0.setString(1, name); - pstmtAddGroup0.setInt(2, flags); - - pstmtAddGroup0.executeUpdate(); - this.conn.commit(); - this.conn.setAutoCommit(true); - this.restarts = 0; // Reset error count - } - catch(SQLException ex) - { - this.conn.rollback(); - this.conn.setAutoCommit(true); - restartConnection(ex); - addGroup(name, flags); - } - } - - public void addEvent(long time, byte type, long gid) - throws SQLException - { - try - { - this.conn.setAutoCommit(false); - this.pstmtAddEvent.setLong(1, time); - this.pstmtAddEvent.setInt(2, type); - this.pstmtAddEvent.setLong(3, gid); - this.pstmtAddEvent.executeUpdate(); - this.conn.commit(); - this.conn.setAutoCommit(true); - this.restarts = 0; - } - catch(SQLException ex) - { - this.conn.rollback(); - this.conn.setAutoCommit(true); - - restartConnection(ex); - addEvent(time, type, gid); - } - } - - public int countArticles() - throws SQLException - { - ResultSet rs = null; - - try - { - rs = this.pstmtCountArticles.executeQuery(); - if(rs.next()) - { - return rs.getInt(1); - } - else - { - return -1; - } - } - catch(SQLException ex) - { - restartConnection(ex); - return countArticles(); - } - finally - { - if(rs != null) - { - rs.close(); - restarts = 0; - } - } - } - - public int countGroups() - throws SQLException - { - ResultSet rs = null; - - try - { - rs = this.pstmtCountGroups.executeQuery(); - if(rs.next()) - { - return rs.getInt(1); - } - else - { - return -1; - } - } - catch(SQLException ex) - { - restartConnection(ex); - return countGroups(); - } - finally - { - if(rs != null) - { - rs.close(); - restarts = 0; - } - } - } - - public void delete(final String messageID) - throws SQLException - { - try - { - this.conn.setAutoCommit(false); - - this.pstmtDeleteArticle0.setString(1, messageID); - int rs = this.pstmtDeleteArticle0.executeUpdate(); - - // We trust the ON DELETE CASCADE functionality to delete - // orphaned references - - this.conn.commit(); - this.conn.setAutoCommit(true); - } - catch(SQLException ex) - { - throw ex; - } - } - - public Article getArticle(String messageID) - throws SQLException - { - ResultSet rs = null; - try - { - pstmtGetArticle0.setString(1, messageID); - rs = pstmtGetArticle0.executeQuery(); - - if(!rs.next()) - { - return null; - } - else - { - String body = new String(rs.getBytes("body")); - String headers = getArticleHeaders(rs.getInt("article_id")); - return new Article(headers, body); - } - } - catch(SQLException ex) - { - restartConnection(ex); - return getArticle(messageID); - } - finally - { - if(rs != null) - { - rs.close(); - restarts = 0; // Reset error count - } - } - } - - /** - * Retrieves an article by its ID. - * @param articleID - * @return - * @throws java.sql.SQLException - */ - public Article getArticle(long articleIndex, long gid) - throws SQLException - { - ResultSet rs = null; - - try - { - this.pstmtGetArticle1.setLong(1, articleIndex); - this.pstmtGetArticle1.setLong(2, gid); - - rs = this.pstmtGetArticle1.executeQuery(); - - if(rs.next()) - { - String body = new String(rs.getBytes("body")); - String headers = getArticleHeaders(rs.getInt("article_id")); - return new Article(headers, body); - } - else - { - return null; - } - } - catch(SQLException ex) - { - restartConnection(ex); - return getArticle(articleIndex, gid); - } - finally - { - if(rs != null) - { - rs.close(); - restarts = 0; - } - } - } - - public String getArticleHeaders(long articleID) - throws SQLException - { - ResultSet rs = null; - - try - { - this.pstmtGetArticleHeaders.setLong(1, articleID); - rs = this.pstmtGetArticleHeaders.executeQuery(); - - StringBuilder buf = new StringBuilder(); - if(rs.next()) - { - for(;;) - { - buf.append(rs.getString(1)); // key - buf.append(": "); - String foldedValue = MimeUtility.fold(0, rs.getString(2)); - buf.append(foldedValue); // value - if(rs.next()) - { - buf.append("\r\n"); - } - else - { - break; - } - } - } - - return buf.toString(); - } - catch(SQLException ex) - { - restartConnection(ex); - return getArticleHeaders(articleID); - } - finally - { - if(rs != null) - rs.close(); - } - } - - public long getArticleIndex(Article article, Group group) - throws SQLException - { - ResultSet rs = null; - - try - { - this.pstmtGetArticleIndex.setString(1, article.getMessageID()); - this.pstmtGetArticleIndex.setLong(2, group.getID()); - - rs = this.pstmtGetArticleIndex.executeQuery(); - if(rs.next()) - { - return rs.getLong(1); - } - else - { - return -1; - } - } - catch(SQLException ex) - { - restartConnection(ex); - return getArticleIndex(article, group); - } - finally - { - if(rs != null) - rs.close(); - } - } - - /** - * Returns a list of Long/Article Pairs. - * @throws java.sql.SQLException - */ - public List> getArticleHeads(Group group, int first, int last) - throws SQLException - { - ResultSet rs = null; - - try - { - this.pstmtGetArticleHeads.setLong(1, group.getID()); - this.pstmtGetArticleHeads.setInt(2, first); - this.pstmtGetArticleHeads.setInt(3, last); - rs = pstmtGetArticleHeads.executeQuery(); - - List> articles - = new ArrayList>(); - - while (rs.next()) - { - long aid = rs.getLong("article_id"); - long aidx = rs.getLong("article_index"); - String headers = getArticleHeaders(aid); - articles.add(new Pair(aidx, - new ArticleHead(headers))); - } - - return articles; - } - catch(SQLException ex) - { - restartConnection(ex); - return getArticleHeads(group, first, last); - } - finally - { - if(rs != null) - rs.close(); - } - } - - public List getArticleNumbers(long gid) - throws SQLException - { - ResultSet rs = null; - try - { - List ids = new ArrayList(); - this.pstmtGetArticleIDs.setLong(1, gid); - rs = this.pstmtGetArticleIDs.executeQuery(); - while(rs.next()) - { - ids.add(rs.getLong(1)); - } - return ids; - } - catch(SQLException ex) - { - restartConnection(ex); - return getArticleNumbers(gid); - } - finally - { - if(rs != null) - { - rs.close(); - restarts = 0; // Clear the restart count after successful request - } - } - } - - public String getConfigValue(String key) - throws SQLException - { - ResultSet rs = null; - try - { - this.pstmtGetConfigValue.setString(1, key); - - rs = this.pstmtGetConfigValue.executeQuery(); - if(rs.next()) - { - return rs.getString(1); // First data on index 1 not 0 - } - else - { - return null; - } - } - catch(SQLException ex) - { - restartConnection(ex); - return getConfigValue(key); - } - finally - { - if(rs != null) - { - rs.close(); - restarts = 0; // Clear the restart count after successful request - } - } - } - - public int getEventsCount(byte type, long start, long end, Group group) - throws SQLException - { - ResultSet rs = null; - - try - { - if(group == null) - { - this.pstmtGetEventsCount0.setInt(1, type); - this.pstmtGetEventsCount0.setLong(2, start); - this.pstmtGetEventsCount0.setLong(3, end); - rs = this.pstmtGetEventsCount0.executeQuery(); - } - else - { - this.pstmtGetEventsCount1.setInt(1, type); - this.pstmtGetEventsCount1.setLong(2, start); - this.pstmtGetEventsCount1.setLong(3, end); - this.pstmtGetEventsCount1.setLong(4, group.getID()); - rs = this.pstmtGetEventsCount1.executeQuery(); - } - - if(rs.next()) - { - return rs.getInt(1); - } - else - { - return -1; - } - } - catch(SQLException ex) - { - restartConnection(ex); - return getEventsCount(type, start, end, group); - } - finally - { - if(rs != null) - rs.close(); - } - } - - /** - * Reads all Groups from the Database. - * @return - * @throws java.sql.SQLException - */ - public List getGroups() - throws SQLException - { - ResultSet rs; - List buffer = new ArrayList(); - Statement stmt = null; - - try - { - stmt = conn.createStatement(); - rs = stmt.executeQuery("SELECT * FROM groups ORDER BY name"); - - while(rs.next()) - { - String name = rs.getString("name"); - long id = rs.getLong("group_id"); - int flags = rs.getInt("flags"); - - Group group = new Group(name, id, flags); - buffer.add(group); - } - - return buffer; - } - catch(SQLException ex) - { - restartConnection(ex); - return getGroups(); - } - finally - { - if(stmt != null) - stmt.close(); // Implicitely closes ResultSets - } - } - - public String getGroupForList(InternetAddress listAddress) - throws SQLException - { - ResultSet rs = null; - - try - { - this.pstmtGetGroupForList.setString(1, listAddress.getAddress()); - - rs = this.pstmtGetGroupForList.executeQuery(); - if (rs.next()) - { - return rs.getString(1); - } - else - { - return null; - } - } - catch(SQLException ex) - { - restartConnection(ex); - return getGroupForList(listAddress); - } - finally - { - if(rs != null) - rs.close(); - } - } - - /** - * Returns the Group that is identified by the name. - * @param name - * @return - * @throws java.sql.SQLException - */ - public Group getGroup(String name) - throws SQLException - { - ResultSet rs = null; - - try - { - this.pstmtGetGroup0.setString(1, name); - rs = this.pstmtGetGroup0.executeQuery(); - - if (!rs.next()) - { - return null; - } - else - { - long id = rs.getLong("group_id"); - int flags = rs.getInt("flags"); - return new Group(name, id, flags); - } - } - catch(SQLException ex) - { - restartConnection(ex); - return getGroup(name); - } - finally - { - if(rs != null) - rs.close(); - } - } - - public String getListForGroup(String group) - throws SQLException - { - ResultSet rs = null; - - try - { - this.pstmtGetListForGroup.setString(1, group); - rs = this.pstmtGetListForGroup.executeQuery(); - if (rs.next()) - { - return rs.getString(1); - } - else - { - return null; - } - } - catch(SQLException ex) - { - restartConnection(ex); - return getListForGroup(group); - } - finally - { - if(rs != null) - rs.close(); - } - } - - private int getMaxArticleIndex(long groupID) - throws SQLException - { - ResultSet rs = null; - - try - { - this.pstmtGetMaxArticleIndex.setLong(1, groupID); - rs = this.pstmtGetMaxArticleIndex.executeQuery(); - - int maxIndex = 0; - if (rs.next()) - { - maxIndex = rs.getInt(1); - } - - return maxIndex; - } - catch(SQLException ex) - { - restartConnection(ex); - return getMaxArticleIndex(groupID); - } - finally - { - if(rs != null) - rs.close(); - } - } - - private int getMaxArticleID() - throws SQLException - { - ResultSet rs = null; - - try - { - rs = this.pstmtGetMaxArticleID.executeQuery(); - - int maxIndex = 0; - if (rs.next()) - { - maxIndex = rs.getInt(1); - } - - return maxIndex; - } - catch(SQLException ex) - { - restartConnection(ex); - return getMaxArticleID(); - } - finally - { - if(rs != null) - rs.close(); - } - } - - public int getLastArticleNumber(Group group) - throws SQLException - { - ResultSet rs = null; - - try - { - this.pstmtGetLastArticleNumber.setLong(1, group.getID()); - rs = this.pstmtGetLastArticleNumber.executeQuery(); - if (rs.next()) - { - return rs.getInt(1); - } - else - { - return 0; - } - } - catch(SQLException ex) - { - restartConnection(ex); - return getLastArticleNumber(group); - } - finally - { - if(rs != null) - rs.close(); - } - } - - public int getFirstArticleNumber(Group group) - throws SQLException - { - ResultSet rs = null; - try - { - this.pstmtGetFirstArticleNumber.setLong(1, group.getID()); - rs = this.pstmtGetFirstArticleNumber.executeQuery(); - if(rs.next()) - { - return rs.getInt(1); - } - else - { - return 0; - } - } - catch(SQLException ex) - { - restartConnection(ex); - return getFirstArticleNumber(group); - } - finally - { - if(rs != null) - rs.close(); - } - } - - /** - * Returns a group name identified by the given id. - * @param id - * @return - * @throws java.sql.SQLException - */ - public String getGroup(int id) - throws SQLException - { - ResultSet rs = null; - - try - { - this.pstmtGetGroup1.setInt(1, id); - rs = this.pstmtGetGroup1.executeQuery(); - - if (rs.next()) - { - return rs.getString(1); - } - else - { - return null; - } - } - catch(SQLException ex) - { - restartConnection(ex); - return getGroup(id); - } - finally - { - if(rs != null) - rs.close(); - } - } - - public double getNumberOfEventsPerHour(int key, long gid) - throws SQLException - { - String gidquery = ""; - if(gid >= 0) - { - gidquery = " AND group_id = " + gid; - } - - Statement stmt = null; - ResultSet rs = null; - - try - { - stmt = this.conn.createStatement(); - rs = stmt.executeQuery("SELECT Count(*) / (Max(event_time) - Min(event_time))" + - " * 1000 * 60 * 60 FROM events WHERE event_key = " + key + gidquery); - - if(rs.next()) - { - restarts = 0; // reset error count - return rs.getDouble(1); - } - else - { - return Double.NaN; - } - } - catch(SQLException ex) - { - restartConnection(ex); - return getNumberOfEventsPerHour(key, gid); - } - finally - { - if(stmt != null) - { - stmt.close(); - } - - if(rs != null) - { - rs.close(); - } - } - } - - public int getPostingsCount(String groupname) - throws SQLException - { - ResultSet rs = null; - - try - { - this.pstmtGetPostingsCount.setString(1, groupname); - rs = this.pstmtGetPostingsCount.executeQuery(); - if(rs.next()) - { - return rs.getInt(1); - } - else - { - Log.msg("Warning: Count on postings return nothing!", true); - return 0; - } - } - catch(SQLException ex) - { - restartConnection(ex); - return getPostingsCount(groupname); - } - finally - { - if(rs != null) - rs.close(); - } - } - - public List getSubscriptions(int feedtype) - throws SQLException - { - ResultSet rs = null; - - try - { - List subs = new ArrayList(); - this.pstmtGetSubscriptions.setInt(1, feedtype); - rs = this.pstmtGetSubscriptions.executeQuery(); - - while(rs.next()) - { - String host = rs.getString("host"); - String group = rs.getString("name"); - int port = rs.getInt("port"); - subs.add(new Subscription(host, port, feedtype, group)); - } - - return subs; - } - catch(SQLException ex) - { - restartConnection(ex); - return getSubscriptions(feedtype); - } - finally - { - if(rs != null) - rs.close(); - } - } - - /** - * Checks if there is an article with the given messageid in the Database. - * @param name - * @return - * @throws java.sql.SQLException - */ - public boolean isArticleExisting(String messageID) - throws SQLException - { - ResultSet rs = null; - - try - { - this.pstmtIsArticleExisting.setString(1, messageID); - rs = this.pstmtIsArticleExisting.executeQuery(); - return rs.next() && rs.getInt(1) == 1; - } - catch(SQLException ex) - { - restartConnection(ex); - return isArticleExisting(messageID); - } - finally - { - if(rs != null) - rs.close(); - } - } - - /** - * Checks if there is a group with the given name in the Database. - * @param name - * @return - * @throws java.sql.SQLException - */ - public boolean isGroupExisting(String name) - throws SQLException - { - ResultSet rs = null; - - try - { - this.pstmtIsGroupExisting.setString(1, name); - rs = this.pstmtIsGroupExisting.executeQuery(); - return rs.next(); - } - catch(SQLException ex) - { - restartConnection(ex); - return isGroupExisting(name); - } - finally - { - if(rs != null) - rs.close(); - } - } - - public void setConfigValue(String key, String value) - throws SQLException - { - try - { - conn.setAutoCommit(false); - this.pstmtSetConfigValue0.setString(1, key); - this.pstmtSetConfigValue0.execute(); - this.pstmtSetConfigValue1.setString(1, key); - this.pstmtSetConfigValue1.setString(2, value); - this.pstmtSetConfigValue1.execute(); - conn.commit(); - conn.setAutoCommit(true); - } - catch(SQLException ex) - { - restartConnection(ex); - setConfigValue(key, value); - } - } - - /** - * Closes the Database connection. - */ - public void shutdown() - throws SQLException - { - if(this.conn != null) - { - this.conn.close(); - } - } - - private void restartConnection(SQLException cause) - throws SQLException - { - restarts++; - Log.msg(Thread.currentThread() - + ": Database connection was closed (restart " + restarts + ").", false); - - if(restarts >= MAX_RESTARTS) - { - // Delete the current, probably broken Database instance. - // So no one can use the instance any more. - Database.instances.remove(Thread.currentThread()); - - // Throw the exception upwards - throw cause; - } - - try - { - Thread.sleep(1500L * restarts); - } - catch(InterruptedException ex) - { - Log.msg("Interrupted: " + ex.getMessage(), false); - } - - // Try to properly close the old database connection - try - { - if(this.conn != null) - { - this.conn.close(); - } - } - catch(SQLException ex) - { - Log.msg(ex.getMessage(), true); - } - - try - { - // Try to reinitialize database connection - arise(); - } - catch(SQLException ex) - { - Log.msg(ex.getMessage(), true); - restartConnection(ex); - } - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/storage/Group.java --- a/org/sonews/daemon/storage/Group.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,186 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.daemon.storage; - -import java.sql.SQLException; -import java.util.List; -import org.sonews.util.Log; -import org.sonews.util.Pair; - -/** - * Represents a logical Group within this newsserver. - * @author Christian Lins - * @since sonews/0.5.0 - */ -public class Group -{ - - /** - * If this flag is set the Group is no real newsgroup but a mailing list - * mirror. In that case every posting and receiving mails must go through - * the mailing list gateway. - */ - public static final int MAILINGLIST = 0x1; - - /** - * If this flag is set the Group is marked as readonly and the posting - * is prohibited. This can be useful for groups that are synced only in - * one direction. - */ - public static final int READONLY = 0x2; - - /** - * If this flag is set the Group is marked as deleted and must not occur - * in any output. The deletion is done lazily by a low priority daemon. - */ - public static final int DELETED = 0x128; - - private long id = 0; - private int flags = -1; - private String name = null; - - /** - * Returns a Group identified by its full name. - * @param name - * @return - */ - public static Group getByName(final String name) - { - try - { - return Database.getInstance().getGroup(name); - } - catch(SQLException ex) - { - ex.printStackTrace(); - return null; - } - } - - /** - * @return List of all groups this server handles. - */ - public static List getAll() - { - try - { - return Database.getInstance().getGroups(); - } - catch(SQLException ex) - { - Log.msg(ex.getMessage(), false); - return null; - } - } - - /** - * Private constructor. - * @param name - * @param id - */ - Group(final String name, final long id, final int flags) - { - this.id = id; - this.flags = flags; - this.name = name; - } - - @Override - public boolean equals(Object obj) - { - if(obj instanceof Group) - { - return ((Group)obj).id == this.id; - } - else - { - return false; - } - } - - public List> getArticleHeads(final int first, final int last) - throws SQLException - { - return Database.getInstance().getArticleHeads(this, first, last); - } - - public List getArticleNumbers() - throws SQLException - { - return Database.getInstance().getArticleNumbers(id); - } - - public int getFirstArticleNumber() - throws SQLException - { - return Database.getInstance().getFirstArticleNumber(this); - } - - /** - * Returns the group id. - */ - public long getID() - { - assert id > 0; - - return id; - } - - public boolean isMailingList() - { - return (this.flags & MAILINGLIST) != 0; - } - - public int getLastArticleNumber() - throws SQLException - { - return Database.getInstance().getLastArticleNumber(this); - } - - public String getName() - { - return name; - } - - /** - * Performs this.flags |= flag to set a specified flag and updates the data - * in the Database. - * @param flag - */ - public void setFlag(final int flag) - { - this.flags |= flag; - } - - public void setName(final String name) - { - this.name = name; - } - - /** - * @return Number of posted articles in this group. - * @throws java.sql.SQLException - */ - public int getPostingsCount() - throws SQLException - { - return Database.getInstance().getPostingsCount(this.name); - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/storage/Headers.java --- a/org/sonews/daemon/storage/Headers.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,51 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.daemon.storage; - -/** - * Contains header constants. These header keys are no way complete but all - * headers that are relevant for sonews. - * @author Christian Lins - * @since sonews/0.5.0 - */ -public final class Headers -{ - - public static final String BYTES = "bytes"; - public static final String CONTENT_TYPE = "content-type"; - public static final String CONTROL = "control"; - public static final String DATE = "date"; - public static final String FROM = "from"; - public static final String LINES = "lines"; - public static final String MESSAGE_ID = "message-id"; - public static final String NEWSGROUPS = "newsgroups"; - public static final String NNTP_POSTING_DATE = "nntp-posting-date"; - public static final String NNTP_POSTING_HOST = "nntp-posting-host"; - public static final String PATH = "path"; - public static final String REFERENCES = "references"; - public static final String SUBJECT = "subject"; - public static final String SUPERSEDES = "subersedes"; - public static final String X_COMPLAINTS_TO = "x-complaints-to"; - public static final String X_TRACE = "x-trace"; - public static final String XREF = "xref"; - - private Headers() - {} - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/daemon/storage/package.html --- a/org/sonews/daemon/storage/package.html Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,2 +0,0 @@ -Contains classes of the storage backend and the Group and Article -abstraction. \ No newline at end of file diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/feed/FeedManager.java --- a/org/sonews/feed/FeedManager.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/feed/FeedManager.java Wed Jul 22 14:04:05 2009 +0200 @@ -18,10 +18,10 @@ package org.sonews.feed; -import java.sql.SQLException; import java.util.List; -import org.sonews.daemon.storage.Article; -import org.sonews.daemon.storage.Database; +import org.sonews.storage.Article; +import org.sonews.storage.StorageBackendException; +import org.sonews.storage.StorageManager; /** * Controlls push and pull feeder. @@ -42,9 +42,9 @@ * PullFeeder or PushFeeder. */ public static synchronized void startFeeding() - throws SQLException + throws StorageBackendException { - List subsPull = Database.getInstance() + List subsPull = StorageManager.current() .getSubscriptions(TYPE_PULL); for(Subscription sub : subsPull) { @@ -52,7 +52,7 @@ } pullFeeder.start(); - List subsPush = Database.getInstance() + List subsPush = StorageManager.current() .getSubscriptions(TYPE_PUSH); for(Subscription sub : subsPush) { diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/feed/PullFeeder.java --- a/org/sonews/feed/PullFeeder.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/feed/PullFeeder.java Wed Jul 22 14:04:05 2009 +0200 @@ -25,14 +25,14 @@ import java.net.Socket; import java.net.SocketException; import java.net.UnknownHostException; -import java.sql.SQLException; import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map; -import org.sonews.daemon.Config; +import org.sonews.config.Config; import org.sonews.util.Log; -import org.sonews.daemon.storage.Database; +import org.sonews.storage.StorageBackendException; +import org.sonews.storage.StorageManager; import org.sonews.util.Stats; import org.sonews.util.io.ArticleReader; import org.sonews.util.io.ArticleWriter; @@ -154,7 +154,7 @@ while(isRunning()) { int pullInterval = 1000 * - Config.getInstance().get(Config.FEED_PULLINTERVAL, 3600); + Config.inst().get(Config.FEED_PULLINTERVAL, 3600); String host = "localhost"; int port = 119; @@ -189,36 +189,34 @@ for(String messageID : messageIDs) { - if(Database.getInstance().isArticleExisting(messageID)) + if(!StorageManager.current().isArticleExisting(messageID)) { - continue; - } - - try - { - // Post the message via common socket connection - ArticleReader aread = - new ArticleReader(sub.getHost(), sub.getPort(), messageID); - byte[] abuf = aread.getArticleData(); - if (abuf == null) + try { - Log.msg("Could not feed " + messageID + " from " + sub.getHost(), true); + // Post the message via common socket connection + ArticleReader aread = + new ArticleReader(sub.getHost(), sub.getPort(), messageID); + byte[] abuf = aread.getArticleData(); + if (abuf == null) + { + Log.msg("Could not feed " + messageID + " from " + sub.getHost(), true); + } + else + { + Log.msg("Feeding " + messageID, true); + ArticleWriter awrite = new ArticleWriter( + "localhost", Config.inst().get(Config.PORT, 119)); + awrite.writeArticle(abuf); + awrite.close(); + } + Stats.getInstance().mailFeeded(sub.getGroup()); } - else + catch(IOException ex) { - Log.msg("Feeding " + messageID, true); - ArticleWriter awrite = new ArticleWriter( - "localhost", Config.getInstance().get(Config.PORT, 119)); - awrite.writeArticle(abuf); - awrite.close(); + // There may be a temporary network failure + ex.printStackTrace(); + Log.msg("Skipping mail " + messageID + " due to exception.", false); } - Stats.getInstance().mailFeeded(sub.getGroup()); - } - catch(IOException ex) - { - // There may be a temporary network failure - ex.printStackTrace(); - Log.msg("Skipping mail " + messageID + " due to exception.", false); } } // for(;;) this.highMarks.put(sub, newMark); @@ -226,7 +224,7 @@ disconnect(); } - catch(SQLException ex) + catch(StorageBackendException ex) { ex.printStackTrace(); } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/feed/PushFeeder.java --- a/org/sonews/feed/PushFeeder.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/feed/PushFeeder.java Wed Jul 22 14:04:05 2009 +0200 @@ -20,8 +20,8 @@ import java.io.IOException; import java.util.concurrent.ConcurrentLinkedQueue; -import org.sonews.daemon.storage.Article; -import org.sonews.daemon.storage.Headers; +import org.sonews.storage.Article; +import org.sonews.storage.Headers; import org.sonews.util.Log; import org.sonews.util.io.ArticleWriter; @@ -90,7 +90,7 @@ } catch(InterruptedException ex) { - Log.msg("PushFeeder interrupted.", true); + Log.msg("PushFeeder interrupted: " + ex, true); } } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/mlgw/Dispatcher.java --- a/org/sonews/mlgw/Dispatcher.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/mlgw/Dispatcher.java Wed Jul 22 14:04:05 2009 +0200 @@ -19,24 +19,19 @@ package org.sonews.mlgw; import java.io.IOException; -import org.sonews.daemon.Config; -import org.sonews.daemon.storage.Article; -import org.sonews.util.io.ArticleInputStream; -import org.sonews.daemon.storage.Database; -import java.sql.SQLException; import java.util.ArrayList; import java.util.List; -import java.util.Properties; import javax.mail.Address; import javax.mail.Authenticator; import javax.mail.Message; import javax.mail.MessagingException; import javax.mail.PasswordAuthentication; -import javax.mail.Session; -import javax.mail.Transport; import javax.mail.internet.InternetAddress; -import javax.mail.internet.MimeMessage; -import org.sonews.daemon.storage.Headers; +import org.sonews.config.Config; +import org.sonews.storage.Article; +import org.sonews.storage.Headers; +import org.sonews.storage.StorageBackendException; +import org.sonews.storage.StorageManager; import org.sonews.util.Log; import org.sonews.util.Stats; @@ -55,9 +50,9 @@ public PasswordAuthentication getPasswordAuthentication() { final String username = - Config.getInstance().get(Config.MLSEND_USER, "user"); + Config.inst().get(Config.MLSEND_USER, "user"); final String password = - Config.getInstance().get(Config.MLSEND_PASSWORD, "mysecret"); + Config.inst().get(Config.MLSEND_PASSWORD, "mysecret"); return new PasswordAuthentication(username, password); } @@ -76,48 +71,71 @@ Address[] to = msg.getAllRecipients(); // includes TO/CC/BCC if(to == null || to.length <= 0) { - Log.msg("Skipping message because no receipient!", true); + to = msg.getReplyTo(); + } + + if(to == null || to.length <= 0) + { + Log.msg("Skipping message because no recipient!", false); return false; } else { - boolean posted = false; - for(Address toa : to) // Address can have '<' '>' around + boolean posted = false; + List newsgroups = new ArrayList(); + + for (Address toa : to) // Address can have '<' '>' around { - if(!(toa instanceof InternetAddress)) + if (toa instanceof InternetAddress) { - continue; + List groups = StorageManager.current() + .getGroupsForList((InternetAddress)toa); + newsgroups.addAll(groups); } - String group = Database.getInstance() - .getGroupForList((InternetAddress)toa); - if(group != null) + } + + if (newsgroups.size() > 0) + { + StringBuilder groups = new StringBuilder(); + for(int n = 0; n < newsgroups.size(); n++) { - Log.msg("Posting to group " + group, true); + groups.append(newsgroups.get(n)); + if(n + 1 != newsgroups.size()) + { + groups.append(','); + } + } + Log.msg("Posting to group " + groups.toString(), true); - // Create new Article object - Article article = new Article(msg); - article.setGroup(group); - - // Write article to database - if(!Database.getInstance().isArticleExisting(article.getMessageID())) - { - Database.getInstance().addArticle(article); - Stats.getInstance().mailGatewayed( - article.getHeader(Headers.NEWSGROUPS)[0]); - } - else - { - Log.msg("Article " + article.getMessageID() + " already existing.", true); - // TODO: It may be possible that a ML mail is posted to several - // ML addresses... - } - posted = true; + // Create new Article object + Article article = new Article(msg); + article.setGroup(groups.toString()); + article.removeHeader(Headers.REPLY_TO); + article.removeHeader(Headers.TO); + + // Write article to database + if(!StorageManager.current().isArticleExisting(article.getMessageID())) + { + StorageManager.current().addArticle(article); + Stats.getInstance().mailGatewayed( + article.getHeader(Headers.NEWSGROUPS)[0]); } else { - Log.msg("No group for " + toa, true); + Log.msg("Article " + article.getMessageID() + " already existing.", true); } - } // end for + posted = true; + } + else + { + StringBuilder buf = new StringBuilder(); + for(Address toa : to) + { + buf.append(' '); + buf.append(toa.toString()); + } + Log.msg("No group for" + buf.toString(), false); + } return posted; } } @@ -132,7 +150,7 @@ * Mails a message received through NNTP to the appropriate mailing list. */ public static void toList(Article article) - throws IOException, MessagingException, SQLException + throws IOException, MessagingException, StorageBackendException { // Get mailing lists for the group of this article List listAddresses = new ArrayList(); @@ -140,7 +158,7 @@ for(String groupname : groupnames) { - String listAddress = Database.getInstance().getListForGroup(groupname); + String listAddress = StorageManager.current().getListForGroup(groupname); if(listAddress != null) { listAddresses.add(listAddress); @@ -150,53 +168,34 @@ for(String listAddress : listAddresses) { // Compose message and send it via given SMTP-Host - String smtpHost = Config.getInstance().get(Config.MLSEND_HOST, "localhost"); - int smtpPort = Config.getInstance().get(Config.MLSEND_PORT, 25); - String smtpUser = Config.getInstance().get(Config.MLSEND_USER, "user"); - String smtpPw = Config.getInstance().get(Config.MLSEND_PASSWORD, "mysecret"); + String smtpHost = Config.inst().get(Config.MLSEND_HOST, "localhost"); + int smtpPort = Config.inst().get(Config.MLSEND_PORT, 25); + String smtpUser = Config.inst().get(Config.MLSEND_USER, "user"); + String smtpPw = Config.inst().get(Config.MLSEND_PASSWORD, "mysecret"); + String smtpFrom = Config.inst().get( + Config.MLSEND_ADDRESS, article.getHeader(Headers.FROM)[0]); - Properties props = System.getProperties(); - props.put("mail.smtp.localhost", - Config.getInstance().get(Config.HOSTNAME, "localhost")); - props.put("mail.smtp.from", // Used for MAIL FROM command - Config.getInstance().get( - Config.MLSEND_ADDRESS, article.getHeader(Headers.FROM)[0])); - props.put("mail.smtp.host", smtpHost); - props.put("mail.smtp.port", smtpPort); - props.put("mail.smtp.auth", "true"); + // TODO: Make Article cloneable() + String group = article.getHeader(Headers.NEWSGROUPS)[0]; + article.getMessageID(); // Make sure an ID is existing + article.removeHeader(Headers.NEWSGROUPS); + article.removeHeader(Headers.PATH); + article.removeHeader(Headers.LINES); + article.removeHeader(Headers.BYTES); - Address[] address = new Address[1]; - address[0] = new InternetAddress(listAddress); + article.setHeader("To", listAddress); + article.setHeader("Reply-To", listAddress); - ArticleInputStream in = new ArticleInputStream(article); - Session session = Session.getDefaultInstance(props, new PasswordAuthenticator()); - MimeMessage msg = new MimeMessage(session, in); - msg.setRecipient(Message.RecipientType.TO, address[0]); - msg.setReplyTo(address); - msg.removeHeader(Headers.NEWSGROUPS); - msg.removeHeader(Headers.PATH); - msg.removeHeader(Headers.LINES); - msg.removeHeader(Headers.BYTES); - - if(Config.getInstance().get(Config.MLSEND_RW_SENDER, false)) + if(Config.inst().get(Config.MLSEND_RW_SENDER, false)) { - rewriteSenderAddress(msg); // Set the SENDER address + rewriteSenderAddress(article); // Set the SENDER address } - - if(Config.getInstance().get(Config.MLSEND_RW_FROM, false)) - { - rewriteFromAddress(msg); // Set the FROM address - } - - msg.saveChanges(); - // Send the mail - Transport transport = session.getTransport("smtp"); - transport.connect(smtpHost, smtpPort, smtpUser, smtpPw); - transport.sendMessage(msg, msg.getAllRecipients()); - transport.close(); + SMTPTransport smtpTransport = new SMTPTransport(smtpHost, smtpPort); + smtpTransport.send(article, smtpFrom, listAddress); + smtpTransport.close(); - Stats.getInstance().mailGatewayed(article.getHeader(Headers.NEWSGROUPS)[0]); + Stats.getInstance().mailGatewayed(group); Log.msg("MLGateway: Mail " + article.getHeader("Subject")[0] + " was delivered to " + listAddress + ".", true); } @@ -208,14 +207,14 @@ * @param msg * @throws javax.mail.MessagingException */ - private static void rewriteSenderAddress(MimeMessage msg) + private static void rewriteSenderAddress(Article msg) throws MessagingException { - String mlAddress = Config.getInstance().get(Config.MLSEND_ADDRESS, null); + String mlAddress = Config.inst().get(Config.MLSEND_ADDRESS, null); if(mlAddress != null) { - msg.setSender(new InternetAddress(mlAddress)); + msg.setHeader(Headers.SENDER, mlAddress); } else { @@ -223,29 +222,4 @@ } } - /** - * Sets the FROM header of the given MimeMessage. This might be necessary - * for moderated groups that does not allow the "normal" FROM sender. - * @param msg - * @throws javax.mail.MessagingException - */ - private static void rewriteFromAddress(MimeMessage msg) - throws MessagingException - { - Address[] froms = msg.getFrom(); - String mlAddress = Config.getInstance().get(Config.MLSEND_ADDRESS, null); - - if(froms.length > 0 && froms[0] instanceof InternetAddress - && mlAddress != null) - { - InternetAddress from = (InternetAddress)froms[0]; - from.setAddress(mlAddress); - msg.setFrom(from); - } - else - { - throw new MessagingException("Cannot rewrite FROM header!"); - } - } - } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/mlgw/MailPoller.java --- a/org/sonews/mlgw/MailPoller.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/mlgw/MailPoller.java Wed Jul 22 14:04:05 2009 +0200 @@ -19,6 +19,7 @@ package org.sonews.mlgw; import java.util.Properties; +import javax.mail.Address; import javax.mail.AuthenticationFailedException; import javax.mail.Authenticator; import javax.mail.Flags.Flag; @@ -29,7 +30,7 @@ import javax.mail.PasswordAuthentication; import javax.mail.Session; import javax.mail.Store; -import org.sonews.daemon.Config; +import org.sonews.config.Config; import org.sonews.daemon.AbstractDaemon; import org.sonews.util.Log; import org.sonews.util.Stats; @@ -49,9 +50,9 @@ public PasswordAuthentication getPasswordAuthentication() { final String username = - Config.getInstance().get(Config.MLPOLL_USER, "user"); + Config.inst().get(Config.MLPOLL_USER, "user"); final String password = - Config.getInstance().get(Config.MLPOLL_PASSWORD, "mysecret"); + Config.inst().get(Config.MLPOLL_PASSWORD, "mysecret"); return new PasswordAuthentication(username, password); } @@ -72,11 +73,11 @@ Thread.sleep(60000 * (errors + 1)); // one minute * errors final String host = - Config.getInstance().get(Config.MLPOLL_HOST, "samplehost"); + Config.inst().get(Config.MLPOLL_HOST, "samplehost"); final String username = - Config.getInstance().get(Config.MLPOLL_USER, "user"); + Config.inst().get(Config.MLPOLL_USER, "user"); final String password = - Config.getInstance().get(Config.MLPOLL_PASSWORD, "mysecret"); + Config.inst().get(Config.MLPOLL_PASSWORD, "mysecret"); Stats.getInstance().mlgwRunStart(); @@ -101,10 +102,8 @@ // Dispatch messages and delete it afterwards on the inbox for(Message message : messages) { - String subject = message.getSubject(); - System.out.println("MLGateway: message with subject \"" + subject + "\" received."); if(Dispatcher.toGroup(message) - || Config.getInstance().get(Config.MLPOLL_DELETEUNKNOWN, false)) + || Config.inst().get(Config.MLPOLL_DELETEUNKNOWN, false)) { // Delete the message message.setFlag(Flag.DELETED, true); @@ -132,7 +131,7 @@ } catch(InterruptedException ex) { - System.out.println("sonews: " + this + " returns."); + System.out.println("sonews: " + this + " returns: " + ex); return; } catch(MessagingException ex) diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/mlgw/SMTPTransport.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/mlgw/SMTPTransport.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,134 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.mlgw; + +import java.io.BufferedOutputStream; +import java.io.BufferedReader; +import java.io.FileOutputStream; +import java.io.IOException; +import java.io.InputStreamReader; +import java.io.PrintWriter; +import java.net.Socket; +import java.net.UnknownHostException; +import org.sonews.config.Config; +import org.sonews.storage.Article; +import org.sonews.util.io.ArticleInputStream; + +/** + * Connects to a SMTP server and sends a given Article to it. + * @author Christian Lins + */ +class SMTPTransport +{ + + protected BufferedReader in; + protected PrintWriter out; + protected Socket socket; + + public SMTPTransport(String host, int port) + throws IOException, UnknownHostException + { + socket = new Socket(host, port); + this.in = new BufferedReader(new InputStreamReader(socket.getInputStream())); + this.out = new PrintWriter(socket.getOutputStream()); + + // Read helo from server + String line = this.in.readLine(); + if(line == null || !line.startsWith("220 ")) + { + throw new IOException("Invalid helo from server: " + line); + } + + // Send HELO to server + this.out.println("HELO " + Config.inst().get(Config.HOSTNAME, "localhost")); + this.out.flush(); + line = this.in.readLine(); + if(line == null || !line.startsWith("250 ")) + { + throw new IOException("Unexpected reply: " + line); + } + } + + public SMTPTransport(String host) + throws IOException + { + this(host, 25); + } + + public void close() + throws IOException + { + this.out.println("QUIT"); + this.out.flush(); + this.in.readLine(); + + this.socket.close(); + } + + public void send(Article article, String mailFrom, String rcptTo) + throws IOException + { + this.out.println("MAIL FROM: " + mailFrom); + this.out.flush(); + String line = this.in.readLine(); + if(line == null || !line.startsWith("250 ")) + { + throw new IOException("Unexpected reply: " + line); + } + + this.out.println("RCPT TO: " + rcptTo); + this.out.flush(); + line = this.in.readLine(); + if(line == null || !line.startsWith("250 ")) + { + throw new IOException("Unexpected reply: " + line); + } + + this.out.println("DATA"); + this.out.flush(); + line = this.in.readLine(); + if(line == null || !line.startsWith("354 ")) + { + throw new IOException("Unexpected reply: " + line); + } + + ArticleInputStream artStream = new ArticleInputStream(article); + BufferedOutputStream outStream = new BufferedOutputStream(socket.getOutputStream()); + FileOutputStream fileStream = new FileOutputStream("smtp.dump"); + for(int b = artStream.read(); b >= 0; b = artStream.read()) + { + outStream.write(b); + fileStream.write(b); + } + + // Flush the binary stream; important because otherwise the output + // will be mixed with the PrintWriter. + outStream.flush(); + fileStream.flush(); + fileStream.close(); + this.out.print("\r\n.\r\n"); + this.out.flush(); + line = this.in.readLine(); + if(line == null || !line.startsWith("250 ")) + { + throw new IOException("Unexpected reply: " + line); + } + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/storage/AggregatedGroup.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/storage/AggregatedGroup.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,260 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.storage; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import org.sonews.util.Pair; + +/** + * An aggregated group is a group consisting of several "real" group. + * @author Christian Lins + * @since sonews/1.0 + */ +class AggregatedGroup extends Channel +{ + + static class GroupElement + { + private Group group; + private long offsetStart, offsetEnd; + + public GroupElement(Group group, long offsetStart, long offsetEnd) + { + this.group = group; + this.offsetEnd = offsetEnd; + this.offsetStart = offsetStart; + } + } + + public static List getAll() + { + List all = new ArrayList(); + all.add(getByName("agg.test")); + return all; + } + + public static AggregatedGroup getByName(String name) + { + if("agg.test".equals(name)) + { + AggregatedGroup agroup = new AggregatedGroup(name); + agroup.addGroup(Group.getByName("agg.test0"), 0, 1000); + agroup.addGroup(Group.getByName("agg.test1"), 2000, 4000); + return agroup; + } + else + return null; + } + + private GroupElement[] groups = new GroupElement[2]; + private String name; + + public AggregatedGroup(String name) + { + this.name = name; + } + + private long aggIdxToIdx(long aggIdx) + throws StorageBackendException + { + assert groups != null && groups.length == 2; + assert groups[0] != null; + assert groups[1] != null; + + // Search in indices of group one + List idxs0 = groups[0].group.getArticleNumbers(); + Collections.sort(idxs0); + for(long idx : idxs0) + { + if(idx == aggIdx) + { + return idx; + } + } + + // Given aggIdx must be an index of group two + List idxs1 = groups[1].group.getArticleNumbers(); + return 0; + } + + private long idxToAggIdx(long idx) + { + return 0; + } + + /** + * Adds the given group to this aggregated set. + * @param group + * @param offsetStart Lower limit for the article ids range + */ + public void addGroup(Group group, long offsetStart, long offsetEnd) + { + this.groups[groups[0] == null ? 0 : 1] + = new GroupElement(group, offsetStart, offsetEnd); + } + + @Override + public Article getArticle(long idx) + throws StorageBackendException + { + Article article = null; + + for(GroupElement groupEl : groups) + { + if(groupEl.offsetStart <= idx && groupEl.offsetEnd >= idx) + { + article = groupEl.group.getArticle(idx - groupEl.offsetStart); + break; + } + } + + return article; + } + + @Override + public List> getArticleHeads( + final long first, final long last) + throws StorageBackendException + { + List> heads = new ArrayList>(); + + for(GroupElement groupEl : groups) + { + List> partHeads = new ArrayList>(); + if(groupEl.offsetStart <= first && groupEl.offsetEnd >= first) + { + long end = Math.min(groupEl.offsetEnd, last); + partHeads = groupEl.group.getArticleHeads + (first - groupEl.offsetStart, end - groupEl.offsetStart); + } + else if(groupEl.offsetStart <= last && groupEl.offsetEnd >= last) + { + long start = Math.max(groupEl.offsetStart, first); + partHeads = groupEl.group.getArticleHeads + (start - groupEl.offsetStart, last - groupEl.offsetStart); + } + + for(Pair partHead : partHeads) + { + heads.add(new Pair( + partHead.getA() + groupEl.offsetStart, partHead.getB())); + } + } + + return heads; + } + + @Override + public List getArticleNumbers() + throws StorageBackendException + { + List articleNumbers = new ArrayList(); + + for(GroupElement groupEl : groups) + { + List partNums = groupEl.group.getArticleNumbers(); + for(Long partNum : partNums) + { + articleNumbers.add(partNum + groupEl.offsetStart); + } + } + + return articleNumbers; + } + + @Override + public long getIndexOf(Article art) + throws StorageBackendException + { + for(GroupElement groupEl : groups) + { + long idx = groupEl.group.getIndexOf(art); + if(idx > 0) + { + return idx; + } + } + return -1; + } + + public long getInternalID() + { + return -1; + } + + @Override + public String getName() + { + return this.name; + } + + @Override + public long getFirstArticleNumber() + throws StorageBackendException + { + long first = Long.MAX_VALUE; + + for(GroupElement groupEl : groups) + { + first = Math.min(first, groupEl.group.getFirstArticleNumber() + groupEl.offsetStart); + } + + return first; + } + + @Override + public long getLastArticleNumber() + throws StorageBackendException + { + long last = 1; + + for(GroupElement groupEl : groups) + { + last = Math.max(last, groupEl.group.getLastArticleNumber() + groupEl.offsetStart); + } + + return last + getPostingsCount(); // This is a hack + } + + public long getPostingsCount() + throws StorageBackendException + { + long postings = 0; + + for(GroupElement groupEl : groups) + { + postings += groupEl.group.getPostingsCount(); + } + + return postings; + } + + public boolean isDeleted() + { + return false; + } + + public boolean isWriteable() + { + return false; + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/storage/Article.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/storage/Article.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,336 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.storage; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.io.InputStream; +import java.nio.charset.Charset; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.util.UUID; +import java.util.ArrayList; +import java.util.Enumeration; +import java.util.List; +import javax.mail.Header; +import javax.mail.Message; +import javax.mail.MessagingException; +import javax.mail.Multipart; +import javax.mail.internet.InternetHeaders; +import org.sonews.config.Config; +import org.sonews.util.Log; + +/** + * Represents a newsgroup article. + * @author Christian Lins + * @author Denis Schwerdel + * @since n3tpd/0.1 + */ +public class Article extends ArticleHead +{ + + /** + * Loads the Article identified by the given ID from the JDBCDatabase. + * @param messageID + * @return null if Article is not found or if an error occurred. + */ + public static Article getByMessageID(final String messageID) + { + try + { + return StorageManager.current().getArticle(messageID); + } + catch(StorageBackendException ex) + { + ex.printStackTrace(); + return null; + } + } + + private byte[] body = new byte[0]; + + /** + * Default constructor. + */ + public Article() + { + } + + /** + * Creates a new Article object using the date from the given + * raw data. + */ + public Article(String headers, byte[] body) + { + try + { + this.body = body; + + // Parse the header + this.headers = new InternetHeaders( + new ByteArrayInputStream(headers.getBytes())); + + this.headerSrc = headers; + } + catch(MessagingException ex) + { + ex.printStackTrace(); + } + } + + /** + * Creates an Article instance using the data from the javax.mail.Message + * object. + * @see javax.mail.Message + * @param msg + * @throws IOException + * @throws MessagingException + */ + public Article(final Message msg) + throws IOException, MessagingException + { + this.headers = new InternetHeaders(); + + for(Enumeration e = msg.getAllHeaders() ; e.hasMoreElements();) + { + final Header header = (Header)e.nextElement(); + this.headers.addHeader(header.getName(), header.getValue()); + } + + // The "content" of the message can be a String if it's a simple text/plain + // message, a Multipart object or an InputStream if the content is unknown. + final Object content = msg.getContent(); + if(content instanceof String) + { + this.body = ((String)content).getBytes(); + } + else if(content instanceof Multipart) // probably subclass MimeMultipart + { + // We're are not interested in the different parts of the MultipartMessage, + // so we simply read in all data which *can* be huge. + InputStream in = msg.getInputStream(); + this.body = readContent(in); + } + else if(content instanceof InputStream) + { + // The message format is unknown to the Message class, but we can + // simply read in the whole message data. + this.body = readContent((InputStream)content); + } + else + { + // Unknown content is probably a malformed mail we should skip. + // On the other hand we produce an inconsistent mail mirror, but no + // mail system must transport invalid content. + Log.msg("Skipping message due to unknown content. Throwing exception...", true); + throw new MessagingException("Unknown content: " + content); + } + + // Validate headers + validateHeaders(); + } + + /** + * Reads from the given InputString into a byte array. + * TODO: Move this generalized method to org.sonews.util.io.Resource. + * @param in + * @return + * @throws IOException + */ + private byte[] readContent(InputStream in) + throws IOException + { + ByteArrayOutputStream out = new ByteArrayOutputStream(); + + int b = in.read(); + while(b >= 0) + { + out.write(b); + b = in.read(); + } + + return out.toByteArray(); + } + + /** + * Removes the header identified by the given key. + * @param headerKey + */ + public void removeHeader(final String headerKey) + { + this.headers.removeHeader(headerKey); + this.headerSrc = null; + } + + /** + * Generates a message id for this article and sets it into + * the header object. You have to update the JDBCDatabase manually to make this + * change persistent. + * Note: a Message-ID should never be changed and only generated once. + */ + private String generateMessageID() + { + String randomString; + MessageDigest md5; + try + { + md5 = MessageDigest.getInstance("MD5"); + md5.reset(); + md5.update(getBody()); + md5.update(getHeader(Headers.SUBJECT)[0].getBytes()); + md5.update(getHeader(Headers.FROM)[0].getBytes()); + byte[] result = md5.digest(); + StringBuffer hexString = new StringBuffer(); + for (int i = 0; i < result.length; i++) + { + hexString.append(Integer.toHexString(0xFF & result[i])); + } + randomString = hexString.toString(); + } + catch (NoSuchAlgorithmException e) + { + e.printStackTrace(); + randomString = UUID.randomUUID().toString(); + } + String msgID = "<" + randomString + "@" + + Config.inst().get(Config.HOSTNAME, "localhost") + ">"; + + this.headers.setHeader(Headers.MESSAGE_ID, msgID); + + return msgID; + } + + /** + * Returns the body string. + */ + public byte[] getBody() + { + return body; + } + + /** + * @return Charset of the body text + */ + private Charset getBodyCharset() + { + // We espect something like + // Content-Type: text/plain; charset=ISO-8859-15 + String contentType = getHeader(Headers.CONTENT_TYPE)[0]; + int idxCharsetStart = contentType.indexOf("charset=") + "charset=".length(); + int idxCharsetEnd = contentType.indexOf(";", idxCharsetStart); + + String charsetName = "UTF-8"; + if(idxCharsetStart >= 0 && idxCharsetStart < contentType.length()) + { + if(idxCharsetEnd < 0) + { + charsetName = contentType.substring(idxCharsetStart); + } + else + { + charsetName = contentType.substring(idxCharsetStart, idxCharsetEnd); + } + } + + // Sometimes there are '"' around the name + if(charsetName.length() > 2 && + charsetName.charAt(0) == '"' && charsetName.endsWith("\"")) + { + charsetName = charsetName.substring(1, charsetName.length() - 2); + } + + // Create charset + Charset charset = Charset.forName("UTF-8"); // This MUST be supported by JVM + try + { + charset = Charset.forName(charsetName); + } + catch(Exception ex) + { + Log.msg(ex.getMessage(), false); + Log.msg("Article.getBodyCharset(): Unknown charset: " + charsetName, false); + } + return charset; + } + + /** + * @return Numerical IDs of the newsgroups this Article belongs to. + */ + public List getGroups() + { + String[] groupnames = getHeader(Headers.NEWSGROUPS)[0].split(","); + ArrayList groups = new ArrayList(); + + try + { + for(String newsgroup : groupnames) + { + newsgroup = newsgroup.trim(); + Group group = StorageManager.current().getGroup(newsgroup); + if(group != null && // If the server does not provide the group, ignore it + !groups.contains(group)) // Yes, there may be duplicates + { + groups.add(group); + } + } + } + catch(StorageBackendException ex) + { + ex.printStackTrace(); + return null; + } + return groups; + } + + public void setBody(byte[] body) + { + this.body = body; + } + + /** + * + * @param groupname Name(s) of newsgroups + */ + public void setGroup(String groupname) + { + this.headers.setHeader(Headers.NEWSGROUPS, groupname); + } + + /** + * Returns the Message-ID of this Article. If the appropriate header + * is empty, a new Message-ID is created. + * @return Message-ID of this Article. + */ + public String getMessageID() + { + String[] msgID = getHeader(Headers.MESSAGE_ID); + return msgID[0].equals("") ? generateMessageID() : msgID[0]; + } + + /** + * @return String containing the Message-ID. + */ + @Override + public String toString() + { + return getMessageID(); + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/storage/ArticleHead.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/storage/ArticleHead.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,161 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.storage; + +import java.io.ByteArrayInputStream; +import java.util.Enumeration; +import javax.mail.Header; +import javax.mail.MessagingException; +import javax.mail.internet.InternetHeaders; +import javax.mail.internet.MimeUtility; +import org.sonews.config.Config; + +/** + * An article with no body only headers. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public class ArticleHead +{ + + protected InternetHeaders headers = null; + protected String headerSrc = null; + + protected ArticleHead() + { + } + + public ArticleHead(String headers) + { + try + { + // Parse the header + this.headers = new InternetHeaders( + new ByteArrayInputStream(headers.getBytes())); + } + catch(MessagingException ex) + { + ex.printStackTrace(); + } + } + + /** + * Returns the header field with given name. + * @param name Name of the header field(s). + * @param returnNull If set to true, this method will return null instead + * of an empty array if there is no header field found. + * @return Header values or empty string. + */ + public String[] getHeader(String name, boolean returnNull) + { + String[] ret = this.headers.getHeader(name); + if(ret == null && !returnNull) + { + ret = new String[]{""}; + } + return ret; + } + + public String[] getHeader(String name) + { + return getHeader(name, false); + } + + /** + * Sets the header value identified through the header name. + * @param name + * @param value + */ + public void setHeader(String name, String value) + { + this.headers.setHeader(name, value); + this.headerSrc = null; + } + + public Enumeration getAllHeaders() + { + return this.headers.getAllHeaders(); + } + + /** + * @return Header source code of this Article. + */ + public String getHeaderSource() + { + if(this.headerSrc != null) + { + return this.headerSrc; + } + + StringBuffer buf = new StringBuffer(); + + for(Enumeration en = this.headers.getAllHeaders(); en.hasMoreElements();) + { + Header entry = (Header)en.nextElement(); + + String value = entry.getValue().replaceAll("[\r\n]", " "); + buf.append(entry.getName()); + buf.append(": "); + buf.append(MimeUtility.fold(entry.getName().length() + 2, value)); + + if(en.hasMoreElements()) + { + buf.append("\r\n"); + } + } + + this.headerSrc = buf.toString(); + return this.headerSrc; + } + + /** + * Sets the headers of this Article. If headers contain no + * Message-Id a new one is created. + * @param headers + */ + public void setHeaders(InternetHeaders headers) + { + this.headers = headers; + this.headerSrc = null; + validateHeaders(); + } + + /** + * Checks some headers for their validity and generates an + * appropriate Path-header for this host if not yet existing. + * This method is called by some Article constructors and the + * method setHeaders(). + * @return true if something on the headers was changed. + */ + protected void validateHeaders() + { + // Check for valid Path-header + final String path = getHeader(Headers.PATH)[0]; + final String host = Config.inst().get(Config.HOSTNAME, "localhost"); + if(!path.startsWith(host)) + { + StringBuffer pathBuf = new StringBuffer(); + pathBuf.append(host); + pathBuf.append('!'); + pathBuf.append(path); + this.headers.setHeader(Headers.PATH, pathBuf.toString()); + } + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/storage/Channel.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/storage/Channel.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,121 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.storage; + +import java.util.ArrayList; +import java.util.List; +import org.sonews.util.Pair; + +/** + * A logical communication Channel is the a generic structural element for sets + * of messages; e.g. a Newsgroup for a set of Articles. + * A Channel can either be a real set of messages or an aggregated set of + * several subsets. + * @author Christian Lins + * @since sonews/1.0 + */ +public abstract class Channel +{ + + /** + * If this flag is set the Group is no real newsgroup but a mailing list + * mirror. In that case every posting and receiving mails must go through + * the mailing list gateway. + */ + public static final int MAILINGLIST = 0x1; + + /** + * If this flag is set the Group is marked as readonly and the posting + * is prohibited. This can be useful for groups that are synced only in + * one direction. + */ + public static final int READONLY = 0x2; + + /** + * If this flag is set the Group is marked as deleted and must not occur + * in any output. The deletion is done lazily by a low priority daemon. + */ + public static final int DELETED = 0x80; + + public static List getAll() + { + List all = new ArrayList(); + + /*List agroups = AggregatedGroup.getAll(); + if(agroups != null) + { + all.addAll(agroups); + }*/ + + List groups = Group.getAll(); + if(groups != null) + { + all.addAll(groups); + } + + return all; + } + + public static Channel getByName(String name) + { + Channel channel; + + // Check if it's an aggregated group + channel = AggregatedGroup.getByName(name); + + // If it's not an aggregate is probably a "real" group + if(channel == null) + { + channel = Group.getByName(name); + } + + return channel; + } + + public abstract Article getArticle(long idx) + throws StorageBackendException; + + public abstract List> getArticleHeads( + final long first, final long last) + throws StorageBackendException; + + public abstract List getArticleNumbers() + throws StorageBackendException; + + public abstract long getFirstArticleNumber() + throws StorageBackendException; + + public abstract long getIndexOf(Article art) + throws StorageBackendException; + + public abstract long getInternalID(); + + public abstract long getLastArticleNumber() + throws StorageBackendException; + + public abstract String getName(); + + public abstract long getPostingsCount() + throws StorageBackendException; + + public abstract boolean isDeleted(); + + public abstract boolean isWriteable(); + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/storage/Group.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/storage/Group.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,202 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.storage; + +import java.sql.SQLException; +import java.util.List; +import org.sonews.util.Log; +import org.sonews.util.Pair; + +/** + * Represents a logical Group within this newsserver. + * @author Christian Lins + * @since sonews/0.5.0 + */ +// TODO: This class should not be public! +public class Group extends Channel +{ + + private long id = 0; + private int flags = -1; + private String name = null; + + /** + * Returns a Group identified by its full name. + * @param name + * @return + */ + public static Group getByName(final String name) + { + try + { + return StorageManager.current().getGroup(name); + } + catch(StorageBackendException ex) + { + ex.printStackTrace(); + return null; + } + } + + /** + * @return List of all groups this server handles. + */ + public static List getAll() + { + try + { + return StorageManager.current().getGroups(); + } + catch(StorageBackendException ex) + { + Log.msg(ex.getMessage(), false); + return null; + } + } + + /** + * @param name + * @param id + */ + public Group(final String name, final long id, final int flags) + { + this.id = id; + this.flags = flags; + this.name = name; + } + + @Override + public boolean equals(Object obj) + { + if(obj instanceof Group) + { + return ((Group)obj).id == this.id; + } + else + { + return false; + } + } + + public Article getArticle(long idx) + throws StorageBackendException + { + return StorageManager.current().getArticle(idx, this.id); + } + + public List> getArticleHeads(final long first, final long last) + throws StorageBackendException + { + return StorageManager.current().getArticleHeads(this, first, last); + } + + public List getArticleNumbers() + throws StorageBackendException + { + return StorageManager.current().getArticleNumbers(id); + } + + public long getFirstArticleNumber() + throws StorageBackendException + { + return StorageManager.current().getFirstArticleNumber(this); + } + + public int getFlags() + { + return this.flags; + } + + public long getIndexOf(Article art) + throws StorageBackendException + { + return StorageManager.current().getArticleIndex(art, this); + } + + /** + * Returns the group id. + */ + public long getInternalID() + { + assert id > 0; + + return id; + } + + public boolean isDeleted() + { + return (this.flags & DELETED) != 0; + } + + public boolean isMailingList() + { + return (this.flags & MAILINGLIST) != 0; + } + + public boolean isWriteable() + { + return true; + } + + public long getLastArticleNumber() + throws StorageBackendException + { + return StorageManager.current().getLastArticleNumber(this); + } + + public String getName() + { + return name; + } + + /** + * Performs this.flags |= flag to set a specified flag and updates the data + * in the JDBCDatabase. + * @param flag + */ + public void setFlag(final int flag) + { + this.flags |= flag; + } + + public void setName(final String name) + { + this.name = name; + } + + /** + * @return Number of posted articles in this group. + * @throws java.sql.SQLException + */ + public long getPostingsCount() + throws StorageBackendException + { + return StorageManager.current().getPostingsCount(this.name); + } + + /** + * Updates flags and name in the backend. + */ + public void update() + throws StorageBackendException + { + StorageManager.current().update(this); + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/storage/Headers.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/storage/Headers.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,54 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.storage; + +/** + * Contains header constants. These header keys are no way complete but all + * headers that are relevant for sonews. + * @author Christian Lins + * @since sonews/0.5.0 + */ +public final class Headers +{ + + public static final String BYTES = "bytes"; + public static final String CONTENT_TYPE = "content-type"; + public static final String CONTROL = "control"; + public static final String DATE = "date"; + public static final String FROM = "from"; + public static final String LINES = "lines"; + public static final String MESSAGE_ID = "message-id"; + public static final String NEWSGROUPS = "newsgroups"; + public static final String NNTP_POSTING_DATE = "nntp-posting-date"; + public static final String NNTP_POSTING_HOST = "nntp-posting-host"; + public static final String PATH = "path"; + public static final String REFERENCES = "references"; + public static final String REPLY_TO = "reply-to"; + public static final String SENDER = "sender"; + public static final String SUBJECT = "subject"; + public static final String SUPERSEDES = "subersedes"; + public static final String TO = "to"; + public static final String X_COMPLAINTS_TO = "x-complaints-to"; + public static final String X_TRACE = "x-trace"; + public static final String XREF = "xref"; + + private Headers() + {} + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/storage/Storage.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/storage/Storage.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,123 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.storage; + +import java.util.List; +import javax.mail.internet.InternetAddress; +import org.sonews.feed.Subscription; +import org.sonews.util.Pair; + +/** + * A generic storage backend interface. + * @author Christian Lins + * @since sonews/1.0 + */ +public interface Storage +{ + + void addArticle(Article art) + throws StorageBackendException; + + void addEvent(long timestamp, int type, long groupID) + throws StorageBackendException; + + void addGroup(String groupname, int flags) + throws StorageBackendException; + + int countArticles() + throws StorageBackendException; + + int countGroups() + throws StorageBackendException; + + void delete(String messageID) + throws StorageBackendException; + + Article getArticle(String messageID) + throws StorageBackendException; + + Article getArticle(long articleIndex, long groupID) + throws StorageBackendException; + + List> getArticleHeads(Group group, long first, long last) + throws StorageBackendException; + + List> getArticleHeaders(Channel channel, long start, long end, + String header, String pattern) + throws StorageBackendException; + + long getArticleIndex(Article art, Group group) + throws StorageBackendException; + + List getArticleNumbers(long groupID) + throws StorageBackendException; + + String getConfigValue(String key) + throws StorageBackendException; + + int getEventsCount(int eventType, long startTimestamp, long endTimestamp, + Channel channel) + throws StorageBackendException; + + double getEventsPerHour(int key, long gid) + throws StorageBackendException; + + int getFirstArticleNumber(Group group) + throws StorageBackendException; + + Group getGroup(String name) + throws StorageBackendException; + + List getGroups() + throws StorageBackendException; + + List getGroupsForList(InternetAddress inetaddress) + throws StorageBackendException; + + int getLastArticleNumber(Group group) + throws StorageBackendException; + + String getListForGroup(String groupname) + throws StorageBackendException; + + String getOldestArticle() + throws StorageBackendException; + + int getPostingsCount(String groupname) + throws StorageBackendException; + + List getSubscriptions(int type) + throws StorageBackendException; + + boolean isArticleExisting(String messageID) + throws StorageBackendException; + + boolean isGroupExisting(String groupname) + throws StorageBackendException; + + void purgeGroup(Group group) + throws StorageBackendException; + + void setConfigValue(String key, String value) + throws StorageBackendException; + + boolean update(Group group) + throws StorageBackendException; + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/storage/StorageBackendException.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/storage/StorageBackendException.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,34 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.storage; + +/** + * + * @author Christian Lins + * @since sonews/1.0 + */ +public class StorageBackendException extends Exception +{ + + public StorageBackendException(Throwable cause) + { + super(cause); + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/storage/StorageManager.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/storage/StorageManager.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,89 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.storage; + +/** + * + * @author Christian Lins + * @since sonews/1.0 + */ +public final class StorageManager +{ + + private static StorageProvider provider; + + public static Storage current() + throws StorageBackendException + { + synchronized(StorageManager.class) + { + if(provider == null) + { + return null; + } + else + { + return provider.storage(Thread.currentThread()); + } + } + } + + public static StorageProvider loadProvider(String pluginClassName) + { + try + { + Class clazz = Class.forName(pluginClassName); + Object inst = clazz.newInstance(); + return (StorageProvider)inst; + } + catch(Exception ex) + { + System.err.println(ex); + return null; + } + } + + /** + * Sets the current storage provider. + * @param provider + */ + public static void enableProvider(StorageProvider provider) + { + synchronized(StorageManager.class) + { + if(StorageManager.provider != null) + { + disableProvider(); + } + StorageManager.provider = provider; + } + } + + /** + * Disables the current provider. + */ + public static void disableProvider() + { + synchronized(StorageManager.class) + { + provider = null; + } + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/storage/StorageProvider.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/storage/StorageProvider.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,40 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.storage; + +/** + * + * @author Christian Lins + * @since sonews/1.0 + */ +public interface StorageProvider +{ + + public boolean isSupported(String uri); + + /** + * This method returns the reference to the associated storage. + * The reference MAY be unique for each thread. In any case it MUST be + * thread-safe to use this method. + * @return The reference to the associated Storage. + */ + public Storage storage(Thread thread) + throws StorageBackendException; + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/storage/impl/JDBCDatabase.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/storage/impl/JDBCDatabase.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,1772 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.storage.impl; + +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.ResultSet; +import java.sql.SQLException; +import java.sql.Statement; +import java.sql.PreparedStatement; +import java.util.ArrayList; +import java.util.Enumeration; +import java.util.List; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import java.util.regex.PatternSyntaxException; +import javax.mail.Header; +import javax.mail.internet.InternetAddress; +import javax.mail.internet.MimeUtility; +import org.sonews.config.Config; +import org.sonews.util.Log; +import org.sonews.feed.Subscription; +import org.sonews.storage.Article; +import org.sonews.storage.ArticleHead; +import org.sonews.storage.Channel; +import org.sonews.storage.Group; +import org.sonews.storage.Storage; +import org.sonews.storage.StorageBackendException; +import org.sonews.util.Pair; + +/** + * JDBCDatabase facade class. + * @author Christian Lins + * @since sonews/0.5.0 + */ +// TODO: Refactor this class to reduce size (e.g. ArticleDatabase GroupDatabase) +public class JDBCDatabase implements Storage +{ + + public static final int MAX_RESTARTS = 3; + + private Connection conn = null; + private PreparedStatement pstmtAddArticle1 = null; + private PreparedStatement pstmtAddArticle2 = null; + private PreparedStatement pstmtAddArticle3 = null; + private PreparedStatement pstmtAddArticle4 = null; + private PreparedStatement pstmtAddGroup0 = null; + private PreparedStatement pstmtAddEvent = null; + private PreparedStatement pstmtCountArticles = null; + private PreparedStatement pstmtCountGroups = null; + private PreparedStatement pstmtDeleteArticle0 = null; + private PreparedStatement pstmtDeleteArticle1 = null; + private PreparedStatement pstmtDeleteArticle2 = null; + private PreparedStatement pstmtDeleteArticle3 = null; + private PreparedStatement pstmtGetArticle0 = null; + private PreparedStatement pstmtGetArticle1 = null; + private PreparedStatement pstmtGetArticleHeaders0 = null; + private PreparedStatement pstmtGetArticleHeaders1 = null; + private PreparedStatement pstmtGetArticleHeads = null; + private PreparedStatement pstmtGetArticleIDs = null; + private PreparedStatement pstmtGetArticleIndex = null; + private PreparedStatement pstmtGetConfigValue = null; + private PreparedStatement pstmtGetEventsCount0 = null; + private PreparedStatement pstmtGetEventsCount1 = null; + private PreparedStatement pstmtGetGroupForList = null; + private PreparedStatement pstmtGetGroup0 = null; + private PreparedStatement pstmtGetGroup1 = null; + private PreparedStatement pstmtGetFirstArticleNumber = null; + private PreparedStatement pstmtGetListForGroup = null; + private PreparedStatement pstmtGetLastArticleNumber = null; + private PreparedStatement pstmtGetMaxArticleID = null; + private PreparedStatement pstmtGetMaxArticleIndex = null; + private PreparedStatement pstmtGetOldestArticle = null; + private PreparedStatement pstmtGetPostingsCount = null; + private PreparedStatement pstmtGetSubscriptions = null; + private PreparedStatement pstmtIsArticleExisting = null; + private PreparedStatement pstmtIsGroupExisting = null; + private PreparedStatement pstmtPurgeGroup0 = null; + private PreparedStatement pstmtPurgeGroup1 = null; + private PreparedStatement pstmtSetConfigValue0 = null; + private PreparedStatement pstmtSetConfigValue1 = null; + private PreparedStatement pstmtUpdateGroup = null; + + /** How many times the database connection was reinitialized */ + private int restarts = 0; + + /** + * Rises the database: reconnect and recreate all prepared statements. + * @throws java.lang.SQLException + */ + protected void arise() + throws SQLException + { + try + { + // Load database driver + Class.forName( + Config.inst().get(Config.STORAGE_DBMSDRIVER, "java.lang.Object")); + + // Establish database connection + this.conn = DriverManager.getConnection( + Config.inst().get(Config.STORAGE_DATABASE, ""), + Config.inst().get(Config.STORAGE_USER, "root"), + Config.inst().get(Config.STORAGE_PASSWORD, "")); + + this.conn.setTransactionIsolation(Connection.TRANSACTION_SERIALIZABLE); + if(this.conn.getTransactionIsolation() != Connection.TRANSACTION_SERIALIZABLE) + { + Log.msg("Warning: Database is NOT fully serializable!", false); + } + + // Prepare statements for method addArticle() + this.pstmtAddArticle1 = conn.prepareStatement( + "INSERT INTO articles (article_id, body) VALUES(?, ?)"); + this.pstmtAddArticle2 = conn.prepareStatement( + "INSERT INTO headers (article_id, header_key, header_value, header_index) " + + "VALUES (?, ?, ?, ?)"); + this.pstmtAddArticle3 = conn.prepareStatement( + "INSERT INTO postings (group_id, article_id, article_index)" + + "VALUES (?, ?, ?)"); + this.pstmtAddArticle4 = conn.prepareStatement( + "INSERT INTO article_ids (article_id, message_id) VALUES (?, ?)"); + + // Prepare statement for method addStatValue() + this.pstmtAddEvent = conn.prepareStatement( + "INSERT INTO events VALUES (?, ?, ?)"); + + // Prepare statement for method addGroup() + this.pstmtAddGroup0 = conn.prepareStatement( + "INSERT INTO groups (name, flags) VALUES (?, ?)"); + + // Prepare statement for method countArticles() + this.pstmtCountArticles = conn.prepareStatement( + "SELECT Count(article_id) FROM article_ids"); + + // Prepare statement for method countGroups() + this.pstmtCountGroups = conn.prepareStatement( + "SELECT Count(group_id) FROM groups WHERE " + + "flags & " + Channel.DELETED + " = 0"); + + // Prepare statements for method delete(article) + this.pstmtDeleteArticle0 = conn.prepareStatement( + "DELETE FROM articles WHERE article_id = " + + "(SELECT article_id FROM article_ids WHERE message_id = ?)"); + this.pstmtDeleteArticle1 = conn.prepareStatement( + "DELETE FROM headers WHERE article_id = " + + "(SELECT article_id FROM article_ids WHERE message_id = ?)"); + this.pstmtDeleteArticle2 = conn.prepareStatement( + "DELETE FROM postings WHERE article_id = " + + "(SELECT article_id FROM article_ids WHERE message_id = ?)"); + this.pstmtDeleteArticle3 = conn.prepareStatement( + "DELETE FROM article_ids WHERE message_id = ?"); + + // Prepare statements for methods getArticle() + this.pstmtGetArticle0 = conn.prepareStatement( + "SELECT * FROM articles WHERE article_id = " + + "(SELECT article_id FROM article_ids WHERE message_id = ?)"); + this.pstmtGetArticle1 = conn.prepareStatement( + "SELECT * FROM articles WHERE article_id = " + + "(SELECT article_id FROM postings WHERE " + + "article_index = ? AND group_id = ?)"); + + // Prepare statement for method getArticleHeaders() + this.pstmtGetArticleHeaders0 = conn.prepareStatement( + "SELECT header_key, header_value FROM headers WHERE article_id = ? " + + "ORDER BY header_index ASC"); + + // Prepare statement for method getArticleHeaders(regular expr pattern) + this.pstmtGetArticleHeaders1 = conn.prepareStatement( + "SELECT p.article_index, h.header_value FROM headers h " + + "INNER JOIN postings p ON h.article_id = p.article_id " + + "INNER JOIN groups g ON p.group_id = g.group_id " + + "WHERE g.name = ? AND " + + "h.header_key = ? AND " + + "p.article_index >= ? " + + "ORDER BY p.article_index ASC"); + + this.pstmtGetArticleIDs = conn.prepareStatement( + "SELECT article_index FROM postings WHERE group_id = ?"); + + // Prepare statement for method getArticleIndex + this.pstmtGetArticleIndex = conn.prepareStatement( + "SELECT article_index FROM postings WHERE " + + "article_id = (SELECT article_id FROM article_ids " + + "WHERE message_id = ?) " + + " AND group_id = ?"); + + // Prepare statements for method getArticleHeads() + this.pstmtGetArticleHeads = conn.prepareStatement( + "SELECT article_id, article_index FROM postings WHERE " + + "postings.group_id = ? AND article_index >= ? AND " + + "article_index <= ?"); + + // Prepare statements for method getConfigValue() + this.pstmtGetConfigValue = conn.prepareStatement( + "SELECT config_value FROM config WHERE config_key = ?"); + + // Prepare statements for method getEventsCount() + this.pstmtGetEventsCount0 = conn.prepareStatement( + "SELECT Count(*) FROM events WHERE event_key = ? AND " + + "event_time >= ? AND event_time < ?"); + + this.pstmtGetEventsCount1 = conn.prepareStatement( + "SELECT Count(*) FROM events WHERE event_key = ? AND " + + "event_time >= ? AND event_time < ? AND group_id = ?"); + + // Prepare statement for method getGroupForList() + this.pstmtGetGroupForList = conn.prepareStatement( + "SELECT name FROM groups INNER JOIN groups2list " + + "ON groups.group_id = groups2list.group_id " + + "WHERE groups2list.listaddress = ?"); + + // Prepare statement for method getGroup() + this.pstmtGetGroup0 = conn.prepareStatement( + "SELECT group_id, flags FROM groups WHERE Name = ?"); + this.pstmtGetGroup1 = conn.prepareStatement( + "SELECT name FROM groups WHERE group_id = ?"); + + // Prepare statement for method getLastArticleNumber() + this.pstmtGetLastArticleNumber = conn.prepareStatement( + "SELECT Max(article_index) FROM postings WHERE group_id = ?"); + + // Prepare statement for method getListForGroup() + this.pstmtGetListForGroup = conn.prepareStatement( + "SELECT listaddress FROM groups2list INNER JOIN groups " + + "ON groups.group_id = groups2list.group_id WHERE name = ?"); + + // Prepare statement for method getMaxArticleID() + this.pstmtGetMaxArticleID = conn.prepareStatement( + "SELECT Max(article_id) FROM articles"); + + // Prepare statement for method getMaxArticleIndex() + this.pstmtGetMaxArticleIndex = conn.prepareStatement( + "SELECT Max(article_index) FROM postings WHERE group_id = ?"); + + // Prepare statement for method getOldestArticle() + this.pstmtGetOldestArticle = conn.prepareStatement( + "SELECT message_id FROM article_ids WHERE article_id = " + + "(SELECT Min(article_id) FROM article_ids)"); + + // Prepare statement for method getFirstArticleNumber() + this.pstmtGetFirstArticleNumber = conn.prepareStatement( + "SELECT Min(article_index) FROM postings WHERE group_id = ?"); + + // Prepare statement for method getPostingsCount() + this.pstmtGetPostingsCount = conn.prepareStatement( + "SELECT Count(*) FROM postings NATURAL JOIN groups " + + "WHERE groups.name = ?"); + + // Prepare statement for method getSubscriptions() + this.pstmtGetSubscriptions = conn.prepareStatement( + "SELECT host, port, name FROM peers NATURAL JOIN " + + "peer_subscriptions NATURAL JOIN groups WHERE feedtype = ?"); + + // Prepare statement for method isArticleExisting() + this.pstmtIsArticleExisting = conn.prepareStatement( + "SELECT Count(article_id) FROM article_ids WHERE message_id = ?"); + + // Prepare statement for method isGroupExisting() + this.pstmtIsGroupExisting = conn.prepareStatement( + "SELECT * FROM groups WHERE name = ?"); + + // Prepare statement for method setConfigValue() + this.pstmtSetConfigValue0 = conn.prepareStatement( + "DELETE FROM config WHERE config_key = ?"); + this.pstmtSetConfigValue1 = conn.prepareStatement( + "INSERT INTO config VALUES(?, ?)"); + + // Prepare statements for method purgeGroup() + this.pstmtPurgeGroup0 = conn.prepareStatement( + "DELETE FROM peer_subscriptions WHERE group_id = ?"); + this.pstmtPurgeGroup1 = conn.prepareStatement( + "DELETE FROM groups WHERE group_id = ?"); + + // Prepare statement for method update(Group) + this.pstmtUpdateGroup = conn.prepareStatement( + "UPDATE groups SET flags = ?, name = ? WHERE group_id = ?"); + } + catch(ClassNotFoundException ex) + { + throw new Error("JDBC Driver not found!", ex); + } + } + + /** + * Adds an article to the database. + * @param article + * @return + * @throws java.sql.SQLException + */ + @Override + public void addArticle(final Article article) + throws StorageBackendException + { + try + { + this.conn.setAutoCommit(false); + + int newArticleID = getMaxArticleID() + 1; + + // Fill prepared statement with values; + // writes body to article table + pstmtAddArticle1.setInt(1, newArticleID); + pstmtAddArticle1.setBytes(2, article.getBody()); + pstmtAddArticle1.execute(); + + // Add headers + Enumeration headers = article.getAllHeaders(); + for(int n = 0; headers.hasMoreElements(); n++) + { + Header header = (Header)headers.nextElement(); + pstmtAddArticle2.setInt(1, newArticleID); + pstmtAddArticle2.setString(2, header.getName().toLowerCase()); + pstmtAddArticle2.setString(3, + header.getValue().replaceAll("[\r\n]", "")); + pstmtAddArticle2.setInt(4, n); + pstmtAddArticle2.execute(); + } + + // For each newsgroup add a reference + List groups = article.getGroups(); + for(Group group : groups) + { + pstmtAddArticle3.setLong(1, group.getInternalID()); + pstmtAddArticle3.setInt(2, newArticleID); + pstmtAddArticle3.setLong(3, getMaxArticleIndex(group.getInternalID()) + 1); + pstmtAddArticle3.execute(); + } + + // Write message-id to article_ids table + this.pstmtAddArticle4.setInt(1, newArticleID); + this.pstmtAddArticle4.setString(2, article.getMessageID()); + this.pstmtAddArticle4.execute(); + + this.conn.commit(); + this.conn.setAutoCommit(true); + + this.restarts = 0; // Reset error count + } + catch(SQLException ex) + { + try + { + this.conn.rollback(); // Rollback changes + } + catch(SQLException ex2) + { + Log.msg("Rollback of addArticle() failed: " + ex2, false); + } + + try + { + this.conn.setAutoCommit(true); // and release locks + } + catch(SQLException ex2) + { + Log.msg("setAutoCommit(true) of addArticle() failed: " + ex2, false); + } + + restartConnection(ex); + addArticle(article); + } + } + + /** + * Adds a group to the JDBCDatabase. This method is not accessible via NNTP. + * @param name + * @throws java.sql.SQLException + */ + @Override + public void addGroup(String name, int flags) + throws StorageBackendException + { + try + { + this.conn.setAutoCommit(false); + pstmtAddGroup0.setString(1, name); + pstmtAddGroup0.setInt(2, flags); + + pstmtAddGroup0.executeUpdate(); + this.conn.commit(); + this.conn.setAutoCommit(true); + this.restarts = 0; // Reset error count + } + catch(SQLException ex) + { + try + { + this.conn.rollback(); + this.conn.setAutoCommit(true); + } + catch(SQLException ex2) + { + ex2.printStackTrace(); + } + + restartConnection(ex); + addGroup(name, flags); + } + } + + @Override + public void addEvent(long time, int type, long gid) + throws StorageBackendException + { + try + { + this.conn.setAutoCommit(false); + this.pstmtAddEvent.setLong(1, time); + this.pstmtAddEvent.setInt(2, type); + this.pstmtAddEvent.setLong(3, gid); + this.pstmtAddEvent.executeUpdate(); + this.conn.commit(); + this.conn.setAutoCommit(true); + this.restarts = 0; + } + catch(SQLException ex) + { + try + { + this.conn.rollback(); + this.conn.setAutoCommit(true); + } + catch(SQLException ex2) + { + ex2.printStackTrace(); + } + + restartConnection(ex); + addEvent(time, type, gid); + } + } + + @Override + public int countArticles() + throws StorageBackendException + { + ResultSet rs = null; + + try + { + rs = this.pstmtCountArticles.executeQuery(); + if(rs.next()) + { + return rs.getInt(1); + } + else + { + return -1; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return countArticles(); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + restarts = 0; + } + } + } + + @Override + public int countGroups() + throws StorageBackendException + { + ResultSet rs = null; + + try + { + rs = this.pstmtCountGroups.executeQuery(); + if(rs.next()) + { + return rs.getInt(1); + } + else + { + return -1; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return countGroups(); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + restarts = 0; + } + } + } + + @Override + public void delete(final String messageID) + throws StorageBackendException + { + try + { + this.conn.setAutoCommit(false); + + this.pstmtDeleteArticle0.setString(1, messageID); + int rs = this.pstmtDeleteArticle0.executeUpdate(); + + // We do not trust the ON DELETE CASCADE functionality to delete + // orphaned references... + this.pstmtDeleteArticle1.setString(1, messageID); + rs = this.pstmtDeleteArticle1.executeUpdate(); + + this.pstmtDeleteArticle2.setString(1, messageID); + rs = this.pstmtDeleteArticle2.executeUpdate(); + + this.pstmtDeleteArticle3.setString(1, messageID); + rs = this.pstmtDeleteArticle3.executeUpdate(); + + this.conn.commit(); + this.conn.setAutoCommit(true); + } + catch(SQLException ex) + { + throw new StorageBackendException(ex); + } + } + + @Override + public Article getArticle(String messageID) + throws StorageBackendException + { + ResultSet rs = null; + try + { + pstmtGetArticle0.setString(1, messageID); + rs = pstmtGetArticle0.executeQuery(); + + if(!rs.next()) + { + return null; + } + else + { + byte[] body = rs.getBytes("body"); + String headers = getArticleHeaders(rs.getInt("article_id")); + return new Article(headers, body); + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getArticle(messageID); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + restarts = 0; // Reset error count + } + } + } + + /** + * Retrieves an article by its ID. + * @param articleID + * @return + * @throws StorageBackendException + */ + @Override + public Article getArticle(long articleIndex, long gid) + throws StorageBackendException + { + ResultSet rs = null; + + try + { + this.pstmtGetArticle1.setLong(1, articleIndex); + this.pstmtGetArticle1.setLong(2, gid); + + rs = this.pstmtGetArticle1.executeQuery(); + + if(rs.next()) + { + byte[] body = rs.getBytes("body"); + String headers = getArticleHeaders(rs.getInt("article_id")); + return new Article(headers, body); + } + else + { + return null; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getArticle(articleIndex, gid); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + restarts = 0; + } + } + } + + /** + * Searches for fitting header values using the given regular expression. + * @param group + * @param start + * @param end + * @param headerKey + * @param pattern + * @return + * @throws StorageBackendException + */ + @Override + public List> getArticleHeaders(Channel group, long start, + long end, String headerKey, String patStr) + throws StorageBackendException, PatternSyntaxException + { + ResultSet rs = null; + List> heads = new ArrayList>(); + + try + { + this.pstmtGetArticleHeaders1.setString(1, group.getName()); + this.pstmtGetArticleHeaders1.setString(2, headerKey); + this.pstmtGetArticleHeaders1.setLong(3, start); + + rs = this.pstmtGetArticleHeaders1.executeQuery(); + + // Convert the "NNTP" regex to Java regex + patStr = patStr.replace("*", ".*"); + Pattern pattern = Pattern.compile(patStr); + + while(rs.next()) + { + Long articleIndex = rs.getLong(1); + if(end < 0 || articleIndex <= end) // Match start is done via SQL + { + String headerValue = rs.getString(2); + Matcher matcher = pattern.matcher(headerValue); + if(matcher.matches()) + { + heads.add(new Pair(articleIndex, headerValue)); + } + } + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getArticleHeaders(group, start, end, headerKey, patStr); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + + return heads; + } + + private String getArticleHeaders(long articleID) + throws StorageBackendException + { + ResultSet rs = null; + + try + { + this.pstmtGetArticleHeaders0.setLong(1, articleID); + rs = this.pstmtGetArticleHeaders0.executeQuery(); + + StringBuilder buf = new StringBuilder(); + if(rs.next()) + { + for(;;) + { + buf.append(rs.getString(1)); // key + buf.append(": "); + String foldedValue = MimeUtility.fold(0, rs.getString(2)); + buf.append(foldedValue); // value + if(rs.next()) + { + buf.append("\r\n"); + } + else + { + break; + } + } + } + + return buf.toString(); + } + catch(SQLException ex) + { + restartConnection(ex); + return getArticleHeaders(articleID); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + @Override + public long getArticleIndex(Article article, Group group) + throws StorageBackendException + { + ResultSet rs = null; + + try + { + this.pstmtGetArticleIndex.setString(1, article.getMessageID()); + this.pstmtGetArticleIndex.setLong(2, group.getInternalID()); + + rs = this.pstmtGetArticleIndex.executeQuery(); + if(rs.next()) + { + return rs.getLong(1); + } + else + { + return -1; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getArticleIndex(article, group); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + /** + * Returns a list of Long/Article Pairs. + * @throws java.sql.SQLException + */ + @Override + public List> getArticleHeads(Group group, long first, + long last) + throws StorageBackendException + { + ResultSet rs = null; + + try + { + this.pstmtGetArticleHeads.setLong(1, group.getInternalID()); + this.pstmtGetArticleHeads.setLong(2, first); + this.pstmtGetArticleHeads.setLong(3, last); + rs = pstmtGetArticleHeads.executeQuery(); + + List> articles + = new ArrayList>(); + + while (rs.next()) + { + long aid = rs.getLong("article_id"); + long aidx = rs.getLong("article_index"); + String headers = getArticleHeaders(aid); + articles.add(new Pair(aidx, + new ArticleHead(headers))); + } + + return articles; + } + catch(SQLException ex) + { + restartConnection(ex); + return getArticleHeads(group, first, last); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + @Override + public List getArticleNumbers(long gid) + throws StorageBackendException + { + ResultSet rs = null; + try + { + List ids = new ArrayList(); + this.pstmtGetArticleIDs.setLong(1, gid); + rs = this.pstmtGetArticleIDs.executeQuery(); + while(rs.next()) + { + ids.add(rs.getLong(1)); + } + return ids; + } + catch(SQLException ex) + { + restartConnection(ex); + return getArticleNumbers(gid); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + restarts = 0; // Clear the restart count after successful request + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + @Override + public String getConfigValue(String key) + throws StorageBackendException + { + ResultSet rs = null; + try + { + this.pstmtGetConfigValue.setString(1, key); + + rs = this.pstmtGetConfigValue.executeQuery(); + if(rs.next()) + { + return rs.getString(1); // First data on index 1 not 0 + } + else + { + return null; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getConfigValue(key); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + restarts = 0; // Clear the restart count after successful request + } + } + } + + @Override + public int getEventsCount(int type, long start, long end, Channel channel) + throws StorageBackendException + { + ResultSet rs = null; + + try + { + if(channel == null) + { + this.pstmtGetEventsCount0.setInt(1, type); + this.pstmtGetEventsCount0.setLong(2, start); + this.pstmtGetEventsCount0.setLong(3, end); + rs = this.pstmtGetEventsCount0.executeQuery(); + } + else + { + this.pstmtGetEventsCount1.setInt(1, type); + this.pstmtGetEventsCount1.setLong(2, start); + this.pstmtGetEventsCount1.setLong(3, end); + this.pstmtGetEventsCount1.setLong(4, channel.getInternalID()); + rs = this.pstmtGetEventsCount1.executeQuery(); + } + + if(rs.next()) + { + return rs.getInt(1); + } + else + { + return -1; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getEventsCount(type, start, end, channel); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + /** + * Reads all Groups from the JDBCDatabase. + * @return + * @throws StorageBackendException + */ + @Override + public List getGroups() + throws StorageBackendException + { + ResultSet rs; + List buffer = new ArrayList(); + Statement stmt = null; + + try + { + stmt = conn.createStatement(); + rs = stmt.executeQuery("SELECT * FROM groups ORDER BY name"); + + while(rs.next()) + { + String name = rs.getString("name"); + long id = rs.getLong("group_id"); + int flags = rs.getInt("flags"); + + Group group = new Group(name, id, flags); + buffer.add(group); + } + + return buffer; + } + catch(SQLException ex) + { + restartConnection(ex); + return getGroups(); + } + finally + { + if(stmt != null) + { + try + { + stmt.close(); // Implicitely closes ResultSets + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + @Override + public List getGroupsForList(InternetAddress listAddress) + throws StorageBackendException + { + ResultSet rs = null; + + try + { + this.pstmtGetGroupForList.setString(1, listAddress.getAddress()); + + rs = this.pstmtGetGroupForList.executeQuery(); + List groups = new ArrayList(); + while(rs.next()) + { + String group = rs.getString(1); + groups.add(group); + } + return groups; + } + catch(SQLException ex) + { + restartConnection(ex); + return getGroupsForList(listAddress); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + /** + * Returns the Group that is identified by the name. + * @param name + * @return + * @throws StorageBackendException + */ + @Override + public Group getGroup(String name) + throws StorageBackendException + { + ResultSet rs = null; + + try + { + this.pstmtGetGroup0.setString(1, name); + rs = this.pstmtGetGroup0.executeQuery(); + + if (!rs.next()) + { + return null; + } + else + { + long id = rs.getLong("group_id"); + int flags = rs.getInt("flags"); + return new Group(name, id, flags); + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getGroup(name); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + @Override + public String getListForGroup(String group) + throws StorageBackendException + { + ResultSet rs = null; + + try + { + this.pstmtGetListForGroup.setString(1, group); + rs = this.pstmtGetListForGroup.executeQuery(); + if (rs.next()) + { + return rs.getString(1); + } + else + { + return null; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getListForGroup(group); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + private int getMaxArticleIndex(long groupID) + throws StorageBackendException + { + ResultSet rs = null; + + try + { + this.pstmtGetMaxArticleIndex.setLong(1, groupID); + rs = this.pstmtGetMaxArticleIndex.executeQuery(); + + int maxIndex = 0; + if (rs.next()) + { + maxIndex = rs.getInt(1); + } + + return maxIndex; + } + catch(SQLException ex) + { + restartConnection(ex); + return getMaxArticleIndex(groupID); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + private int getMaxArticleID() + throws StorageBackendException + { + ResultSet rs = null; + + try + { + rs = this.pstmtGetMaxArticleID.executeQuery(); + + int maxIndex = 0; + if (rs.next()) + { + maxIndex = rs.getInt(1); + } + + return maxIndex; + } + catch(SQLException ex) + { + restartConnection(ex); + return getMaxArticleID(); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + @Override + public int getLastArticleNumber(Group group) + throws StorageBackendException + { + ResultSet rs = null; + + try + { + this.pstmtGetLastArticleNumber.setLong(1, group.getInternalID()); + rs = this.pstmtGetLastArticleNumber.executeQuery(); + if (rs.next()) + { + return rs.getInt(1); + } + else + { + return 0; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getLastArticleNumber(group); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + @Override + public int getFirstArticleNumber(Group group) + throws StorageBackendException + { + ResultSet rs = null; + try + { + this.pstmtGetFirstArticleNumber.setLong(1, group.getInternalID()); + rs = this.pstmtGetFirstArticleNumber.executeQuery(); + if(rs.next()) + { + return rs.getInt(1); + } + else + { + return 0; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getFirstArticleNumber(group); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + /** + * Returns a group name identified by the given id. + * @param id + * @return + * @throws StorageBackendException + */ + public String getGroup(int id) + throws StorageBackendException + { + ResultSet rs = null; + + try + { + this.pstmtGetGroup1.setInt(1, id); + rs = this.pstmtGetGroup1.executeQuery(); + + if (rs.next()) + { + return rs.getString(1); + } + else + { + return null; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getGroup(id); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + @Override + public double getEventsPerHour(int key, long gid) + throws StorageBackendException + { + String gidquery = ""; + if(gid >= 0) + { + gidquery = " AND group_id = " + gid; + } + + Statement stmt = null; + ResultSet rs = null; + + try + { + stmt = this.conn.createStatement(); + rs = stmt.executeQuery("SELECT Count(*) / (Max(event_time) - Min(event_time))" + + " * 1000 * 60 * 60 FROM events WHERE event_key = " + key + gidquery); + + if(rs.next()) + { + restarts = 0; // reset error count + return rs.getDouble(1); + } + else + { + return Double.NaN; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getEventsPerHour(key, gid); + } + finally + { + try + { + if(stmt != null) + { + stmt.close(); // Implicitely closes the result sets + } + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + + @Override + public String getOldestArticle() + throws StorageBackendException + { + ResultSet rs = null; + + try + { + rs = this.pstmtGetOldestArticle.executeQuery(); + if(rs.next()) + { + return rs.getString(1); + } + else + { + return null; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getOldestArticle(); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + @Override + public int getPostingsCount(String groupname) + throws StorageBackendException + { + ResultSet rs = null; + + try + { + this.pstmtGetPostingsCount.setString(1, groupname); + rs = this.pstmtGetPostingsCount.executeQuery(); + if(rs.next()) + { + return rs.getInt(1); + } + else + { + Log.msg("Warning: Count on postings return nothing!", true); + return 0; + } + } + catch(SQLException ex) + { + restartConnection(ex); + return getPostingsCount(groupname); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + @Override + public List getSubscriptions(int feedtype) + throws StorageBackendException + { + ResultSet rs = null; + + try + { + List subs = new ArrayList(); + this.pstmtGetSubscriptions.setInt(1, feedtype); + rs = this.pstmtGetSubscriptions.executeQuery(); + + while(rs.next()) + { + String host = rs.getString("host"); + String group = rs.getString("name"); + int port = rs.getInt("port"); + subs.add(new Subscription(host, port, feedtype, group)); + } + + return subs; + } + catch(SQLException ex) + { + restartConnection(ex); + return getSubscriptions(feedtype); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + /** + * Checks if there is an article with the given messageid in the JDBCDatabase. + * @param name + * @return + * @throws StorageBackendException + */ + @Override + public boolean isArticleExisting(String messageID) + throws StorageBackendException + { + ResultSet rs = null; + + try + { + this.pstmtIsArticleExisting.setString(1, messageID); + rs = this.pstmtIsArticleExisting.executeQuery(); + return rs.next() && rs.getInt(1) == 1; + } + catch(SQLException ex) + { + restartConnection(ex); + return isArticleExisting(messageID); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + /** + * Checks if there is a group with the given name in the JDBCDatabase. + * @param name + * @return + * @throws StorageBackendException + */ + @Override + public boolean isGroupExisting(String name) + throws StorageBackendException + { + ResultSet rs = null; + + try + { + this.pstmtIsGroupExisting.setString(1, name); + rs = this.pstmtIsGroupExisting.executeQuery(); + return rs.next(); + } + catch(SQLException ex) + { + restartConnection(ex); + return isGroupExisting(name); + } + finally + { + if(rs != null) + { + try + { + rs.close(); + } + catch(SQLException ex) + { + ex.printStackTrace(); + } + } + } + } + + @Override + public void setConfigValue(String key, String value) + throws StorageBackendException + { + try + { + conn.setAutoCommit(false); + this.pstmtSetConfigValue0.setString(1, key); + this.pstmtSetConfigValue0.execute(); + this.pstmtSetConfigValue1.setString(1, key); + this.pstmtSetConfigValue1.setString(2, value); + this.pstmtSetConfigValue1.execute(); + conn.commit(); + conn.setAutoCommit(true); + } + catch(SQLException ex) + { + restartConnection(ex); + setConfigValue(key, value); + } + } + + /** + * Closes the JDBCDatabase connection. + */ + public void shutdown() + throws StorageBackendException + { + try + { + if(this.conn != null) + { + this.conn.close(); + } + } + catch(SQLException ex) + { + throw new StorageBackendException(ex); + } + } + + @Override + public void purgeGroup(Group group) + throws StorageBackendException + { + try + { + this.pstmtPurgeGroup0.setLong(1, group.getInternalID()); + this.pstmtPurgeGroup0.executeUpdate(); + + this.pstmtPurgeGroup1.setLong(1, group.getInternalID()); + this.pstmtPurgeGroup1.executeUpdate(); + } + catch(SQLException ex) + { + restartConnection(ex); + purgeGroup(group); + } + } + + private void restartConnection(SQLException cause) + throws StorageBackendException + { + restarts++; + Log.msg(Thread.currentThread() + + ": Database connection was closed (restart " + restarts + ").", false); + + if(restarts >= MAX_RESTARTS) + { + // Delete the current, probably broken JDBCDatabase instance. + // So no one can use the instance any more. + JDBCDatabaseProvider.instances.remove(Thread.currentThread()); + + // Throw the exception upwards + throw new StorageBackendException(cause); + } + + try + { + Thread.sleep(1500L * restarts); + } + catch(InterruptedException ex) + { + Log.msg("Interrupted: " + ex.getMessage(), false); + } + + // Try to properly close the old database connection + try + { + if(this.conn != null) + { + this.conn.close(); + } + } + catch(SQLException ex) + { + Log.msg(ex.getMessage(), true); + } + + try + { + // Try to reinitialize database connection + arise(); + } + catch(SQLException ex) + { + Log.msg(ex.getMessage(), true); + restartConnection(ex); + } + } + + /** + * Writes the flags and the name of the given group to the database. + * @param group + * @throws StorageBackendException + */ + @Override + public boolean update(Group group) + throws StorageBackendException + { + try + { + this.pstmtUpdateGroup.setInt(1, group.getFlags()); + this.pstmtUpdateGroup.setString(2, group.getName()); + this.pstmtUpdateGroup.setLong(3, group.getInternalID()); + int rs = this.pstmtUpdateGroup.executeUpdate(); + return rs == 1; + } + catch(SQLException ex) + { + restartConnection(ex); + return update(group); + } + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/storage/impl/JDBCDatabaseProvider.java --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/storage/impl/JDBCDatabaseProvider.java Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,67 @@ +/* + * SONEWS News Server + * see AUTHORS for the list of contributors + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package org.sonews.storage.impl; + +import org.sonews.storage.*; +import java.sql.SQLException; +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; + +/** + * + * @author Christian Lins + * @since sonews/1.0 + */ +public class JDBCDatabaseProvider implements StorageProvider +{ + + protected static final Map instances + = new ConcurrentHashMap(); + + @Override + public boolean isSupported(String uri) + { + throw new UnsupportedOperationException("Not supported yet."); + } + + @Override + public Storage storage(Thread thread) + throws StorageBackendException + { + try + { + if(!instances.containsKey(Thread.currentThread())) + { + JDBCDatabase db = new JDBCDatabase(); + db.arise(); + instances.put(Thread.currentThread(), db); + return db; + } + else + { + return instances.get(Thread.currentThread()); + } + } + catch(SQLException ex) + { + throw new StorageBackendException(ex); + } + } + +} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/storage/package.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/org/sonews/storage/package.html Wed Jul 22 14:04:05 2009 +0200 @@ -0,0 +1,2 @@ +Contains classes of the storage backend and the Group and Article +abstraction. \ No newline at end of file diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/util/AbstractConfig.java --- a/org/sonews/util/AbstractConfig.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,43 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.util; - -/** - * Base class for Config and BootstrapConfig. - * @author Christian Lins - * @since sonews/0.5.0 - */ -public abstract class AbstractConfig -{ - - public abstract String get(String key, String defVal); - - public int get(final String key, final int defVal) - { - return Integer.parseInt( - get(key, Integer.toString(defVal))); - } - - public boolean get(String key, boolean defVal) - { - String val = get(key, Boolean.toString(defVal)); - return Boolean.parseBoolean(val); - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/util/DatabaseSetup.java --- a/org/sonews/util/DatabaseSetup.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/util/DatabaseSetup.java Wed Jul 22 14:04:05 2009 +0200 @@ -25,7 +25,7 @@ import java.sql.Statement; import java.util.HashMap; import java.util.Map; -import org.sonews.daemon.BootstrapConfig; +import org.sonews.config.Config; import org.sonews.util.io.Resource; /** @@ -108,7 +108,9 @@ for(String chunk : tmplChunks) { if(chunk.trim().equals("")) + { continue; + } Statement stmt = conn.createStatement(); stmt.execute(chunk); @@ -117,12 +119,7 @@ conn.commit(); conn.setAutoCommit(true); - BootstrapConfig config = BootstrapConfig.getInstance(); - config.set(BootstrapConfig.STORAGE_DATABASE, url); - config.set(BootstrapConfig.STORAGE_DBMSDRIVER, driverMap.get(dbmsType)); - config.set(BootstrapConfig.STORAGE_PASSWORD, dbPassword); - config.set(BootstrapConfig.STORAGE_USER, dbUser); - config.save(); + // Create config file System.out.println("Ok"); } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/util/Log.java --- a/org/sonews/util/Log.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/util/Log.java Wed Jul 22 14:04:05 2009 +0200 @@ -18,8 +18,8 @@ package org.sonews.util; -import org.sonews.daemon.*; import java.util.Date; +import org.sonews.config.Config; /** * Provides logging and debugging methods. @@ -31,10 +31,10 @@ public static boolean isDebug() { - // We must use BootstrapConfig here otherwise we come + // We must use FileConfig here otherwise we come // into hell's kittchen when using the Logger within the // Database class. - return BootstrapConfig.getInstance().get(BootstrapConfig.DEBUG, false); + return Config.inst().get(Config.DEBUG, false); } /** diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/util/Purger.java --- a/org/sonews/util/Purger.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/util/Purger.java Wed Jul 22 14:04:05 2009 +0200 @@ -18,70 +18,131 @@ package org.sonews.util; -import org.sonews.daemon.Config; -import org.sonews.daemon.storage.Database; -import org.sonews.daemon.storage.Article; +import org.sonews.daemon.AbstractDaemon; +import org.sonews.config.Config; +import org.sonews.storage.Article; +import org.sonews.storage.Headers; import java.util.Date; +import java.util.List; +import org.sonews.storage.Channel; +import org.sonews.storage.Group; +import org.sonews.storage.StorageBackendException; +import org.sonews.storage.StorageManager; /** * The purger is started in configurable intervals to search - * for old messages that can be purged. + * for messages that can be purged. A message must be deleted if its lifetime + * has exceeded, if it was marked as deleted or if the maximum number of + * articles in the database is reached. * @author Christian Lins * @since sonews/0.5.0 */ -public class Purger +public class Purger extends AbstractDaemon { - private long lifetime; - - public Purger() - { - this.lifetime = Config.getInstance().get("sonews.article.lifetime", 30) - * 24L * 60L * 60L * 1000L; // in Milliseconds - } - /** * Loops through all messages and deletes them if their time * has come. */ - void purge() - throws Exception + @Override + public void run() { - System.out.println("Purging old messages..."); + try + { + while(isRunning()) + { + purgeDeleted(); + purgeOutdated(); - for (;;) + Thread.sleep(120000); // Sleep for two minutes + } + } + catch(StorageBackendException ex) { - // TODO: Delete articles directly in database - Article art = null; //Database.getInstance().getOldestArticle(); - if (art == null) // No articles in the database + ex.printStackTrace(); + } + catch(InterruptedException ex) + { + Log.msg("Purger interrupted: " + ex, true); + } + } + + private void purgeDeleted() + throws StorageBackendException + { + List groups = StorageManager.current().getGroups(); + for(Channel channel : groups) + { + if(!(channel instanceof Group)) + continue; + + Group group = (Group)channel; + // Look for groups that are marked as deleted + if(group.isDeleted()) { - break; + List ids = StorageManager.current().getArticleNumbers(group.getInternalID()); + if(ids.size() == 0) + { + StorageManager.current().purgeGroup(group); + Log.msg("Group " + group.getName() + " purged.", true); + } + + for(int n = 0; n < ids.size() && n < 10; n++) + { + Article art = StorageManager.current().getArticle(ids.get(n), group.getInternalID()); + StorageManager.current().delete(art.getMessageID()); + Log.msg("Article " + art.getMessageID() + " purged.", true); + } + } + } + } + + private void purgeOutdated() + throws InterruptedException, StorageBackendException + { + long articleMaximum = + Config.inst().get("sonews.article.maxnum", Long.MAX_VALUE); + long lifetime = + Config.inst().get("sonews.article.lifetime", -1); + + if(lifetime > 0 || articleMaximum < Stats.getInstance().getNumberOfNews()) + { + Log.msg("Purging old messages...", true); + String mid = StorageManager.current().getOldestArticle(); + if (mid == null) // No articles in the database + { + return; } -/* if (art.getDate().getTime() < (new Date().getTime() + this.lifetime)) + Article art = StorageManager.current().getArticle(mid); + long artDate = 0; + String dateStr = art.getHeader(Headers.DATE)[0]; + try { - // Database.getInstance().delete(art); - System.out.println("Deleted: " + art); + artDate = Date.parse(dateStr) / 1000 / 60 / 60 / 24; + } + catch (IllegalArgumentException ex) + { + Log.msg("Could not parse date string: " + dateStr + " " + ex, true); + } + + // Should we delete the message because of its age or because the + // article maximum was reached? + if (lifetime < 0 || artDate < (new Date().getTime() + lifetime)) + { + StorageManager.current().delete(mid); + System.out.println("Deleted: " + mid); } else { - break; - }*/ + Thread.sleep(1000 * 60); // Wait 60 seconds + return; + } } - } - - public static void main(String[] args) - { - try + else { - Purger purger = new Purger(); - purger.purge(); - System.exit(0); - } - catch(Exception ex) - { - ex.printStackTrace(); - System.exit(1); + Log.msg("Lifetime purger is disabled", true); + Thread.sleep(1000 * 60 * 30); // Wait 30 minutes } } diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/util/Stats.java --- a/org/sonews/util/Stats.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/util/Stats.java Wed Jul 22 14:04:05 2009 +0200 @@ -18,10 +18,11 @@ package org.sonews.util; -import java.sql.SQLException; import java.util.Calendar; -import org.sonews.daemon.storage.Database; -import org.sonews.daemon.storage.Group; +import org.sonews.config.Config; +import org.sonews.storage.Channel; +import org.sonews.storage.StorageBackendException; +import org.sonews.storage.StorageManager; /** * Class that capsulates statistical data gathering. @@ -48,26 +49,36 @@ private Stats() {} private volatile int connectedClients = 0; - + + /** + * A generic method that writes event data to the storage backend. + * If event logging is disabled with sonews.eventlog=false this method + * simply does nothing. + * @param type + * @param groupname + */ private void addEvent(byte type, String groupname) { - Group group = Group.getByName(groupname); - if(group != null) + if(Config.inst().get(Config.EVENTLOG, true)) { - try + Channel group = Channel.getByName(groupname); + if(group != null) { - Database.getInstance().addEvent( - System.currentTimeMillis(), type, group.getID()); + try + { + StorageManager.current().addEvent( + System.currentTimeMillis(), type, group.getInternalID()); + } + catch(StorageBackendException ex) + { + ex.printStackTrace(); + } } - catch(SQLException ex) + else { - ex.printStackTrace(); + Log.msg("Group " + groupname + " does not exist.", true); } } - else - { - Log.msg("Group " + groupname + " does not exist.", true); - } } public void clientConnect() @@ -89,9 +100,9 @@ { try { - return Database.getInstance().countGroups(); + return StorageManager.current().countGroups(); } - catch(SQLException ex) + catch(StorageBackendException ex) { ex.printStackTrace(); return -1; @@ -102,9 +113,9 @@ { try { - return Database.getInstance().countArticles(); + return StorageManager.current().countArticles(); } - catch(SQLException ex) + catch(StorageBackendException ex) { ex.printStackTrace(); return -1; @@ -112,7 +123,7 @@ } public int getYesterdaysEvents(final byte eventType, final int hour, - final Group group) + final Channel group) { // Determine the timestamp values for yesterday and the given hour Calendar cal = Calendar.getInstance(); @@ -128,10 +139,10 @@ try { - return Database.getInstance() + return StorageManager.current() .getEventsCount(eventType, startTimestamp, endTimestamp, group); } - catch(SQLException ex) + catch(StorageBackendException ex) { ex.printStackTrace(); return -1; @@ -167,9 +178,9 @@ { try { - return Database.getInstance().getNumberOfEventsPerHour(key, gid); + return StorageManager.current().getEventsPerHour(key, gid); } - catch(SQLException ex) + catch(StorageBackendException ex) { ex.printStackTrace(); return -1; diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/util/io/ArticleInputStream.java --- a/org/sonews/util/io/ArticleInputStream.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/util/io/ArticleInputStream.java Wed Jul 22 14:04:05 2009 +0200 @@ -20,9 +20,9 @@ import java.io.ByteArrayOutputStream; import java.io.IOException; -import org.sonews.daemon.storage.*; import java.io.InputStream; import java.io.UnsupportedEncodingException; +import org.sonews.storage.Article; /** * Capsulates an Article to provide a raw InputStream. @@ -41,11 +41,12 @@ final ByteArrayOutputStream out = new ByteArrayOutputStream(); out.write(art.getHeaderSource().getBytes("UTF-8")); out.write("\r\n\r\n".getBytes()); - out.write(art.getBody().getBytes(art.getBodyCharset())); + out.write(art.getBody()); // Without CRLF out.flush(); this.buffer = out.toByteArray(); } - + + @Override public int read() { if(offset >= buffer.length) diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/util/io/ArticleReader.java --- a/org/sonews/util/io/ArticleReader.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/util/io/ArticleReader.java Wed Jul 22 14:04:05 2009 +0200 @@ -26,6 +26,7 @@ import java.io.UnsupportedEncodingException; import java.net.Socket; import java.net.UnknownHostException; +import org.sonews.config.Config; import org.sonews.util.Log; /** @@ -72,6 +73,8 @@ public byte[] getArticleData() throws IOException, UnsupportedEncodingException { + long maxSize = Config.inst().get(Config.ARTICLE_MAXSIZE, 1024) * 1024L; + try { this.out.write(("ARTICLE " + this.messageID + "\r\n").getBytes("UTF-8")); @@ -90,6 +93,11 @@ } buf.write(10); + if(buf.size() > maxSize) + { + Log.msg("Skipping message that is too large: " + buf.size(), false); + return null; + } } return buf.toByteArray(); diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/util/io/ArticleWriter.java --- a/org/sonews/util/io/ArticleWriter.java Wed Jul 01 10:48:22 2009 +0200 +++ b/org/sonews/util/io/ArticleWriter.java Wed Jul 22 14:04:05 2009 +0200 @@ -25,7 +25,7 @@ import java.io.UnsupportedEncodingException; import java.net.Socket; import java.net.UnknownHostException; -import org.sonews.daemon.storage.Article; +import org.sonews.storage.Article; /** * Posts an Article to a NNTP server using the POST command. diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/util/io/VarCharsetReader.java --- a/org/sonews/util/io/VarCharsetReader.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,90 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.util.io; - -import java.io.IOException; -import java.io.InputStream; -import java.nio.ByteBuffer; -import java.nio.charset.Charset; - -/** - * InputStream that can change its decoding charset while reading from the - * underlying byte based stream. - * @author Christian Lins - * @since sonews/0.5.0 - */ -public class VarCharsetReader -{ - - private final ByteBuffer buf = ByteBuffer.allocate(4096); - private InputStream in; - - public VarCharsetReader(final InputStream in) - { - if(in == null) - { - throw new IllegalArgumentException("null InputStream"); - } - this.in = in; - } - - /** - * Reads up to the next newline character and returns the line as String. - * The String is decoded using the given charset. - */ - public String readLine(Charset charset) - throws IOException - { - byte[] byteBuf = new byte[1]; - String bufStr; - - for(;;) - { - int read = this.in.read(byteBuf); - if(read == 0) - { - continue; - } - else if(read == -1) - { - this.in = null; - bufStr = new String(this.buf.array(), 0, this.buf.position(), charset); - break; - } - else if(byteBuf[0] == 10) // Is this safe? \n - { - bufStr = new String(this.buf.array(), 0, this.buf.position(), charset); - break; - } - else if(byteBuf[0] == 13) // \r - { // Skip - continue; - } - else - { - this.buf.put(byteBuf[0]); - } - } - - this.buf.clear(); - - return bufStr; - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/AbstractSonewsServlet.java --- a/org/sonews/web/AbstractSonewsServlet.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,113 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.web; - -import java.io.BufferedReader; -import java.io.IOException; -import java.io.InputStreamReader; -import java.io.PrintWriter; -import java.net.Socket; -import javax.servlet.http.HttpServlet; -import org.sonews.util.StringTemplate; -import org.sonews.util.io.Resource; - -/** - * Base class for all sonews servlets. - * @author Christian Lins - * @since sonews/0.5.0 - */ -public class AbstractSonewsServlet extends HttpServlet -{ - - public static final String TemplateRoot = "org/sonews/web/tmpl/"; - - protected String hello = null; - - private BufferedReader in = null; - private PrintWriter out = null; - private Socket socket = null; - - protected void connectToNewsserver() - throws IOException - { - // Get sonews port from properties - String port = System.getProperty("sonews.port", "9119"); - String host = System.getProperty("sonews.host", "localhost"); - - try - { - this.socket = new Socket(host, Integer.parseInt(port)); - - this.in = new BufferedReader( - new InputStreamReader(socket.getInputStream())); - this.out = new PrintWriter(socket.getOutputStream()); - - hello = in.readLine(); // Read hello message - } - catch(IOException ex) - { - System.out.println("sonews.host=" + host); - System.out.println("sonews.port=" + port); - System.out.flush(); - throw ex; - } - } - - protected void disconnectFromNewsserver() - { - try - { - printlnToNewsserver("QUIT"); - out.close(); - readlnFromNewsserver(); // Wait for bye message - in.close(); - socket.close(); - } - catch(IOException ex) - { - ex.printStackTrace(); - } - } - - protected StringTemplate getTemplate(String res) - { - StringTemplate tmpl = new StringTemplate( - Resource.getAsString(TemplateRoot + "AbstractSonewsServlet.tmpl", true)); - String content = Resource.getAsString(TemplateRoot + res, true); - String stylesheet = System.getProperty("sonews.web.stylesheet", "style.css"); - - tmpl.set("CONTENT", content); - tmpl.set("STYLESHEET", stylesheet); - - return new StringTemplate(tmpl.toString()); - } - - protected void printlnToNewsserver(final String line) - { - this.out.println(line); - this.out.flush(); - } - - protected String readlnFromNewsserver() - throws IOException - { - return this.in.readLine(); - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/MemoryBitmapChart.java --- a/org/sonews/web/MemoryBitmapChart.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,61 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.web; - -import info.monitorenter.gui.chart.Chart2D; -import info.monitorenter.gui.chart.IAxis.AxisTitle; -import java.awt.Color; -import java.awt.image.BufferedImage; -import java.io.ByteArrayOutputStream; -import java.io.IOException; -import javax.imageio.ImageIO; - -/** - * A chart rendered to a memory bitmap. - * @author Christian Lins - * @since sonews/0.5.0 - */ -class MemoryBitmapChart extends Chart2D -{ - - public MemoryBitmapChart() - { - setGridColor(Color.LIGHT_GRAY); - getAxisX().setPaintGrid(true); - getAxisY().setPaintGrid(true); - getAxisX().setAxisTitle(new AxisTitle("time of day")); - getAxisY().setAxisTitle(new AxisTitle("processed news")); - } - - public String getContentType() - { - return "image/png"; - } - - public byte[] getRawData(final int width, final int height) - throws IOException - { - setSize(width, height); - BufferedImage img = snapShot(width, height); - ByteArrayOutputStream out = new ByteArrayOutputStream(); - ImageIO.write(img, "png", out); - return out.toByteArray(); - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/SonewsChartServlet.java --- a/org/sonews/web/SonewsChartServlet.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,114 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.web; - -import info.monitorenter.gui.chart.ITrace2D; -import info.monitorenter.gui.chart.traces.Trace2DSimple; -import java.io.IOException; -import javax.servlet.http.HttpServletRequest; -import javax.servlet.http.HttpServletResponse; - -/** - * Servlet that creates chart images and returns them as raw PNG images. - * @author Christian Lins - * @since sonews/0.5.0 - */ -public class SonewsChartServlet extends AbstractSonewsServlet -{ - - private ITrace2D createProcessMails24(String title, String cmd) - throws IOException - { - int[] data = read24Values(cmd); - ITrace2D trace = new Trace2DSimple(title); - trace.addPoint(0.0, 0.0); // Start - - for(int n = 0; n < 24; n++) - { - trace.addPoint(n, data[n]); - } - - return trace; - } - - @Override - public void doGet(HttpServletRequest req, HttpServletResponse resp) - throws IOException - { - synchronized(this) - { - MemoryBitmapChart chart = new MemoryBitmapChart(); - - String name = req.getParameter("name"); - String group = req.getParameter("group"); - ITrace2D trace; - String cmd = "XDAEMON LOG"; - - if(name.equals("feedednewsyesterday")) - { - cmd = cmd + " TRANSMITTED_NEWS"; - cmd = group != null ? cmd + " " + group : cmd; - trace = createProcessMails24( - "To peers transmitted mails yesterday", cmd); - } - else if(name.equals("gatewayednewsyesterday")) - { - cmd = cmd + " GATEWAYED_NEWS"; - cmd = group != null ? cmd + " " + group : cmd; - trace = createProcessMails24( - "Gatewayed mails yesterday", cmd); - } - else - { - cmd = cmd + " POSTED_NEWS"; - cmd = group != null ? cmd + " " + group : cmd; - trace = createProcessMails24( - "Posted mails yesterday", cmd); - } - chart.addTrace(trace); - - resp.getOutputStream().write(chart.getRawData(500, 400)); - resp.setContentType(chart.getContentType()); - resp.setStatus(HttpServletResponse.SC_OK); - } - } - - private int[] read24Values(String command) - throws IOException - { - int[] values = new int[24]; - connectToNewsserver(); - printlnToNewsserver(command); - String line = readlnFromNewsserver(); - if(!line.startsWith("200 ")) - throw new IOException(command + " not supported!"); - - for(int n = 0; n < 24; n++) - { - line = readlnFromNewsserver(); - values[n] = Integer.parseInt(line.split(" ")[1]); - } - - line = readlnFromNewsserver(); // "." - - disconnectFromNewsserver(); - return values; - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/SonewsConfigServlet.java --- a/org/sonews/web/SonewsConfigServlet.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,239 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.web; - -import java.io.IOException; -import java.util.ArrayList; -import java.util.List; -import java.util.Set; -import javax.servlet.http.HttpServletRequest; -import javax.servlet.http.HttpServletResponse; -import org.sonews.util.StringTemplate; -import org.sonews.util.io.Resource; - -/** - * Servlet providing a configuration web interface. - * @author Christian Lins - * @since sonews/0.5.0 - */ -public class SonewsConfigServlet extends AbstractSonewsServlet -{ - - private static final long serialVersionUID = 2432543253L; - - @Override - public void doGet(HttpServletRequest req, HttpServletResponse resp) - throws IOException - { - synchronized(this) - { - connectToNewsserver(); - String which = req.getParameter("which"); - - if(which != null && which.equals("config")) - { - whichConfig(req, resp); - } - else if(which != null && which.equals("groupadd")) - { - whichGroupAdd(req, resp); - } - else if(which != null && which.equals("groupdelete")) - { - whichGroupDelete(req, resp); - } - else - { - whichNone(req, resp); - } - - disconnectFromNewsserver(); - } - } - - private void whichConfig(HttpServletRequest req, HttpServletResponse resp) - throws IOException - { - StringBuilder keys = new StringBuilder(); - - Set pnames = req.getParameterMap().keySet(); - for(Object obj : pnames) - { - String pname = (String)obj; - if(pname.startsWith("configkey:")) - { - String value = req.getParameter(pname); - String key = pname.split(":")[1]; - if(!value.equals("")) - { - printlnToNewsserver("XDAEMON SET " + key + " " + value); - readlnFromNewsserver(); - - keys.append(key); - keys.append("
"); - } - } - } - - StringTemplate tmpl = getTemplate("ConfigUpdated.tmpl"); - - tmpl.set("UPDATED_KEYS", keys.toString()); - - resp.setStatus(HttpServletResponse.SC_OK); - resp.getWriter().println(tmpl.toString()); - resp.getWriter().flush(); - } - - private void whichGroupAdd(HttpServletRequest req, HttpServletResponse resp) - throws IOException - { - String[] groupnames = req.getParameter("groups").split("\n"); - - for(String groupname : groupnames) - { - groupname = groupname.trim(); - if(groupname.equals("")) - { - continue; - } - - printlnToNewsserver("XDAEMON GROUPADD " + groupname + " 0"); - String line = readlnFromNewsserver(); - if(!line.startsWith("200 ")) - { - System.out.println("Warning " + groupname + " probably not created!"); - } - } - - StringTemplate tmpl = getTemplate("GroupAdded.tmpl"); - - tmpl.set("GROUP", req.getParameter("groups")); - - resp.setStatus(HttpServletResponse.SC_OK); - resp.getWriter().println(tmpl.toString()); - resp.getWriter().flush(); - } - - private void whichGroupDelete(HttpServletRequest req, HttpServletResponse resp) - throws IOException - { - String groupname = req.getParameter("group"); - printlnToNewsserver("XDAEMON GROUPDEL " + groupname); - String line = readlnFromNewsserver(); - if(!line.startsWith("200 ")) - throw new IOException(line); - - StringTemplate tmpl = getTemplate("GroupDeleted.tmpl"); - - tmpl.set("GROUP", groupname); - - resp.setStatus(HttpServletResponse.SC_OK); - resp.getWriter().println(tmpl.toString()); - resp.getWriter().flush(); - } - - private void whichNone(HttpServletRequest req, HttpServletResponse resp) - throws IOException - { - StringTemplate tmpl = getTemplate("SonewsConfigServlet.tmpl"); - - // Retrieve config keys from server - List configKeys = new ArrayList(); - printlnToNewsserver("XDAEMON LIST CONFIGKEYS"); - String line = readlnFromNewsserver(); - if(!line.startsWith("200 ")) - throw new IOException("XDAEMON command not supported!"); - for(;;) - { - line = readlnFromNewsserver(); - if(line.equals(".")) - break; - else - configKeys.add(line); - } - - // Construct config table - StringBuilder strb = new StringBuilder(); - for(String key : configKeys) - { - strb.append(""); - strb.append(key); - strb.append(""); - - // Retrieve config value from server - String value = ""; - printlnToNewsserver("XDAEMON GET " + key); - line = readlnFromNewsserver(); - if(line.startsWith("200 ")) - { - value = readlnFromNewsserver(); - readlnFromNewsserver(); // Read the "." - } - - strb.append(""); - } - tmpl.set("CONFIG", strb.toString()); - - // Retrieve served newsgroup names from server - List groups = new ArrayList(); - printlnToNewsserver("LIST"); - line = readlnFromNewsserver(); - if(line.startsWith("215 ")) - { - for(;;) - { - line = readlnFromNewsserver(); - if(line.equals(".")) - { - break; - } - else - { - groups.add(line.split(" ")[0]); - } - } - } - else - throw new IOException("Error issuing LIST command!"); - - // Construct groups list - StringTemplate tmplGroupList = new StringTemplate( - Resource.getAsString("org/sonews/web/tmpl/GroupList.tmpl", true)); - strb = new StringBuilder(); - for(String group : groups) - { - tmplGroupList.set("GROUPNAME", group); - strb.append(tmplGroupList.toString()); - } - tmpl.set("GROUP", strb.toString()); - - // Set server name - tmpl.set("SERVERNAME", hello.split(" ")[2]); - tmpl.set("TITLE", "Configuration"); - - resp.getWriter().println(tmpl.toString()); - resp.getWriter().flush(); - resp.setStatus(HttpServletResponse.SC_OK); - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/SonewsGroupServlet.java --- a/org/sonews/web/SonewsGroupServlet.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,66 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.web; - -import java.io.IOException; -import javax.servlet.http.HttpServletRequest; -import javax.servlet.http.HttpServletResponse; -import org.sonews.util.StringTemplate; - -/** - * Views the group settings and allows editing. - * @author Christian Lins - * @since sonews/0.5.0 - */ -public class SonewsGroupServlet extends AbstractSonewsServlet -{ - - @Override - public void doGet(HttpServletRequest req, HttpServletResponse resp) - throws IOException - { - synchronized(this) - { - connectToNewsserver(); - String name = req.getParameter("name"); - String action = req.getParameter("action"); - - if("set_flags".equals(action)) - { - - } - else if("set_mladdress".equals(action)) - { - - } - - StringTemplate tmpl = getTemplate("SonewsGroupServlet.tmpl"); - tmpl.set("SERVERNAME", hello.split(" ")[2]); - tmpl.set("TITLE", "Group " + name); - tmpl.set("GROUPNAME", name); - - resp.getWriter().println(tmpl.toString()); - resp.getWriter().flush(); - resp.setStatus(HttpServletResponse.SC_OK); - - disconnectFromNewsserver(); - } - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/SonewsPeerServlet.java --- a/org/sonews/web/SonewsPeerServlet.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,95 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.web; - -import java.io.IOException; -import java.util.HashSet; -import javax.servlet.http.HttpServletRequest; -import javax.servlet.http.HttpServletResponse; -import org.sonews.util.StringTemplate; - -/** - * Servlet that shows the Peers and the Peering Rules. - * @author Christian Lins - * @since sonews/0.5.0 - */ -public class SonewsPeerServlet extends AbstractSonewsServlet -{ - - private static final long serialVersionUID = 245345346356L; - - @Override - public void doGet(HttpServletRequest req, HttpServletResponse resp) - throws IOException - { - synchronized(this) - { - connectToNewsserver(); - StringTemplate tmpl = getTemplate("SonewsPeerServlet.tmpl"); - - // Read peering rules from newsserver - printlnToNewsserver("XDAEMON LIST PEERINGRULES"); - String line = readlnFromNewsserver(); - if(!line.startsWith("200 ")) - { - throw new IOException("Unexpected reply: " + line); - } - - // Create FEED_RULES String - HashSet peers = new HashSet(); - StringBuilder feedRulesStr = new StringBuilder(); - for(;;) - { - line = readlnFromNewsserver(); - if(line.equals(".")) - { - break; - } - else - { - feedRulesStr.append(line); - feedRulesStr.append("
"); - - String[] lineChunks = line.split(" "); - peers.add(lineChunks[1]); - } - } - - // Create PEERS string - StringBuilder peersStr = new StringBuilder(); - for(String peer : peers) - { - peersStr.append(peer); - peersStr.append("
"); - } - - // Set server name - tmpl.set("PEERS", peersStr.toString()); - tmpl.set("PEERING_RULES", feedRulesStr.toString()); - tmpl.set("SERVERNAME", hello.split(" ")[2]); - tmpl.set("TITLE", "Peers"); - - resp.getWriter().println(tmpl.toString()); - resp.getWriter().flush(); - resp.setStatus(HttpServletResponse.SC_OK); - disconnectFromNewsserver(); - } - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/SonewsServlet.java --- a/org/sonews/web/SonewsServlet.java Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,127 +0,0 @@ -/* - * SONEWS News Server - * see AUTHORS for the list of contributors - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -package org.sonews.web; - -import java.io.IOException; -import javax.servlet.http.HttpServletRequest; -import javax.servlet.http.HttpServletResponse; -import org.sonews.daemon.Main; -import org.sonews.util.StringTemplate; - -/** - * Main sonews webpage servlet. - * @author Christian Lins - * @since sonews/0.5.0 - */ -public class SonewsServlet extends AbstractSonewsServlet -{ - - private static final long serialVersionUID = 2392837459834L; - - @Override - public void doGet(HttpServletRequest res, HttpServletResponse resp) - throws IOException - { - synchronized(this) - { - connectToNewsserver(); - - String line; - int connectedClients = 0; - int hostedGroups = 0; - int hostedNews = 0; - - printlnToNewsserver("XDAEMON LOG CONNECTED_CLIENTS"); - - line = readlnFromNewsserver(); - if(!line.startsWith("200 ")) - { - throw new IOException("XDAEMON command not allowed by server"); - } - line = readlnFromNewsserver(); - connectedClients = Integer.parseInt(line); - line = readlnFromNewsserver(); // Read the "." - - printlnToNewsserver("XDAEMON LOG HOSTED_NEWS"); - line = readlnFromNewsserver(); - if(!line.startsWith("200 ")) - { - throw new IOException("XDAEMON command not allowed by server"); - } - line = readlnFromNewsserver(); - hostedNews = Integer.parseInt(line); - line = readlnFromNewsserver(); // read the "." - - printlnToNewsserver("XDAEMON LOG HOSTED_GROUPS"); - line = readlnFromNewsserver(); - if(!line.startsWith("200 ")) - { - throw new IOException("XDAEMON command not allowed by server"); - } - line = readlnFromNewsserver(); - hostedGroups = Integer.parseInt(line); - line = readlnFromNewsserver(); // read the "." - - printlnToNewsserver("XDAEMON LOG POSTED_NEWS_PER_HOUR"); - line = readlnFromNewsserver(); - if(!line.startsWith("200 ")) - { - throw new IOException("XDAEMON command not allowed by server"); - } - String postedNewsPerHour = readlnFromNewsserver(); - readlnFromNewsserver(); - - printlnToNewsserver("XDAEMON LOG GATEWAYED_NEWS_PER_HOUR"); - line = readlnFromNewsserver(); - if(!line.startsWith("200 ")) - { - throw new IOException("XDAEMON command not allowed by server"); - } - String gatewayedNewsPerHour = readlnFromNewsserver(); - line = readlnFromNewsserver(); - - printlnToNewsserver("XDAEMON LOG FEEDED_NEWS_PER_HOUR"); - line = readlnFromNewsserver(); - if(!line.startsWith("200 ")) - { - throw new IOException("XDAEMON command not allowed by server"); - } - String feededNewsPerHour = readlnFromNewsserver(); - line = readlnFromNewsserver(); - - StringTemplate tmpl = getTemplate("SonewsServlet.tmpl"); - tmpl.set("SERVERNAME", hello.split(" ")[2]); - tmpl.set("STARTDATE", Main.STARTDATE); - tmpl.set("ACTIVE_CONNECTIONS", connectedClients); - tmpl.set("STORED_NEWS", hostedNews); - tmpl.set("SERVED_NEWSGROUPS", hostedGroups); - tmpl.set("POSTED_NEWS", postedNewsPerHour); - tmpl.set("GATEWAYED_NEWS", gatewayedNewsPerHour); - tmpl.set("FEEDED_NEWS", feededNewsPerHour); - tmpl.set("TITLE", "Overview"); - - resp.getWriter().println(tmpl.toString()); - resp.getWriter().flush(); - resp.setStatus(HttpServletResponse.SC_OK); - - disconnectFromNewsserver(); - } - } - -} diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/package.html --- a/org/sonews/web/package.html Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,3 +0,0 @@ -Contains classes of the sonews web interface. These classes are not needed by -the running sonews daemon but by the Servlet container -Kitten. \ No newline at end of file diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/tmpl/AbstractSonewsServlet.tmpl --- a/org/sonews/web/tmpl/AbstractSonewsServlet.tmpl Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,14 +0,0 @@ - - - %SERVERNAME - %TITLE - - - - -
sonews - %TITLE
- -
-%CONTENT -
- - \ No newline at end of file diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/tmpl/ConfigUpdated.tmpl --- a/org/sonews/web/tmpl/ConfigUpdated.tmpl Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,7 +0,0 @@ -

- The following config keys were updated:
- %UPDATED_KEYS -

-

- Back to Config -

\ No newline at end of file diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/tmpl/GroupAdded.tmpl --- a/org/sonews/web/tmpl/GroupAdded.tmpl Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,6 +0,0 @@ -

- The Newsgroup %GROUP has been created! -

-

- Back to Config -

\ No newline at end of file diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/tmpl/GroupDeleted.tmpl --- a/org/sonews/web/tmpl/GroupDeleted.tmpl Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,6 +0,0 @@ -

- The Newsgroup %GROUP and all associated articles have been deleted! -

-

- Back to Config -

\ No newline at end of file diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/tmpl/GroupList.tmpl --- a/org/sonews/web/tmpl/GroupList.tmpl Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,8 +0,0 @@ - - - %GROUPNAME - - - delete - - \ No newline at end of file diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/tmpl/SonewsConfigServlet.tmpl --- a/org/sonews/web/tmpl/SonewsConfigServlet.tmpl Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,30 +0,0 @@ -

-Back to Main Page -

- -

Configuration values

-
- - -%CONFIG -
- -
- -

Groups served by this sonews instance

- -%GROUP -
- -

-

Add new group to be served

-
- - - - - -
Names (separated by newlines):
- -
-

\ No newline at end of file diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/tmpl/SonewsGroupServlet.tmpl --- a/org/sonews/web/tmpl/SonewsGroupServlet.tmpl Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,36 +0,0 @@ -

-Back to Main Page -

- -

Group %GROUPNAME

- -

Configuration

-

General

-
- - - Is mirrored Mailinglist?: - -
- -
- -

Mailinglist

-
- - - Mailinglist address: - -
- -
- -

Statistics

-

Posted mails yesterday

- - -

Gatewayed mails yesterday

- - -

Feeded news

- \ No newline at end of file diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/tmpl/SonewsPeerServlet.tmpl --- a/org/sonews/web/tmpl/SonewsPeerServlet.tmpl Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,52 +0,0 @@ -

-Back to Main Page -

- -On this page you can configure to which peer hosts new messages are -posted or from which peer hosts we should receive messages. - -

Peers

-%PEERS - -

Add new peer

-
- - - - - - - - - -
Hostname:
Port:
- -
- -

Rules

-

Current

-%PEERING_RULES - -

Add peering rule

-
- - - - - - - - - - - - - -
Peer:%OPTION_PEERS
Group:%OPTION_GROUP
Peering type: - -
- -
\ No newline at end of file diff -r 1090e2141798 -r 2fdc9cc89502 org/sonews/web/tmpl/SonewsServlet.tmpl --- a/org/sonews/web/tmpl/SonewsServlet.tmpl Wed Jul 01 10:48:22 2009 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,54 +0,0 @@ -This server is running since %STARTDATE. - -

Configuration

-Here you can edit most of the configuration values for %SERVERNAME. -Please note that some of these values require a server restart, -some do not and are valid immediately. - - -

Statistics & Logs

-Here is a short overview of useful statistics of %SERVERNAME. Click on a -stat key to get more details. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Stat keyValue
Active connections:%ACTIVE_CONNECTIONS
Served newsgroups:%SERVED_NEWSGROUPS
Stored news messages:%STORED_NEWS
Posted news:%POSTED_NEWS per hour
Gatewayed news:%GATEWAYED_NEWS per hour
Feeded news:%FEEDED_NEWS per hour
- -

Documentation

-You'll find the most recent -documentation of %SERVERNAME here. \ No newline at end of file