twitter: @crashpolaroid pics or ba (updated 1 hour ago)
With all eyes on Phil Hellmuth, who highlighted a tough final table of the World Poker Tour Bay 101 Shooting Star Final table, where the players vied for $878,500 first place title, it was the least likely player, McLean Karr who took down the...
I Don't Believe in Imaginary Property writes "PJ of Groklaw has found some really interesting documents coming out of the never-ending SCO trial. Specifically, in SCO v. Novell, SCO doesn't want the jury to find out about the email Blake Stowell (then a PR guy for SCO) sent to Maureen O'Gara that asked her to 'send a jab PJ's way.' For those who don't remember that far back in the SCO saga, the 'jab' was when O'Gara wrote an inaccurate, rambling and irrelevant 'exposé' on PJ which got O'Gara fired for violating journalistic ethics after angry readers complained to the publisher—an act which caused Ms. O'Gara to tell SCO, 'I want war pay.' For those wondering how they can keep going after that final judgment against SCO over a year ago, it's hard to do the saga justice without glossing over everything, but the short version is that SCO ran to bankruptcy after they were mostly dead, but before becoming completely dead. That automatically stopped all the cases against SCO due to standard bankruptcy court rules, then SCO effectively re-litigated a bunch of issues via bankruptcy court rules. Currently, they're accusing Novell of 'slander of title' over copyrights that two different courts have ruled SCO does not own, and we're waiting to see if a jury will reach the same conclusion. They're also trying to use the company's lawsuits as assets and to sell them to various SCO insiders so that the legal wranglings can continue even if nothing is left of SCO. From the very start, SCO has always been the type to fight dirty."
Read more of this story at Slashdot.
Don't miss the $2,000 and $500 PokerNews Cash Freerolls going on at Betfred Poker. These cash freerolls are only available when you sign up to Betfred Poker through PokerNews and make a minimum $50 deposit. Take advantage of the remaining...
holmesfsf writes "Creeped out by the Lower Merion School District's remote monitoring of students? Check out the Free Software Foundation's response to the laptop spying scandal and help build a wiki listing of school districts that provide students with laptops, so that the FSF can campaign against mandatory, proprietary laptops."
Read more of this story at Slashdot.
Barence writes "Beneath the web pages indexed by Google lies an online world that few know exists. It's a realm of huge, untapped reserves of valuable information containing sprawling databases, hidden websites and murky forums. It's a world where academics and researchers might find the data required to solve some of mankind's biggest problems, but also where criminal syndicates operate, and terrorist handbooks and child pornography are freely distributed. Interested? You're not alone. The deep web and its 'darknets' are a new battleground for those who want to uphold the right to privacy online, and those who feel that rights need to be sacrificed for the safety of society. The deep web is also the new frontier for those who want to rival Google in the field of search." The melodrama is tempered, though: "The deep web isn’t half as strange or sinister as it sounds. In computer-science speak, it refers to those portions of the web that, for whatever reason, have been invisible to conventional search engines such as Google."
Read more of this story at Slashdot.
MMBK writes "Dennis Chamberland is one of the world's preeminent aquanauts. He's worked with NASA to develop living habitats and underwater plant growth labs, among other cool things. His next goal is establishing the world's first permanent underwater colony. This video gets to the heart of his project, literally and figuratively, as most is shot in his underwater habitat, Atlantica, off the coast of Key Largo, FL. The coolest part might be the moon pool, the room you swim into underwater."
Read more of this story at Slashdot.
Muff so soft shampoo
Picture by: dunno source Submitted by: rg3 via Fail Uploader
superapecommando writes "GCHQ lost 35 laptops in one year, potentially containing highly sensitive data. The UK's electronic spy centre was today lambasted by MPs for having a 'cavalier' attitude to data security. The centre is responsible for tracking the electronic communications of terrorists. In a new report, the Commons Intelligence and Security Committee expressed concern that GCHQ appeared to be entirely unaware whether or not the computers, lost in 2008, contained top secret information on people posing an imminent security threat to the country."
Read more of this story at Slashdot.
Earthquake Retrofit writes "The Register has a rather funny story about the Zeus botnet: 'The latest version of the Zeus do-it-yourself crimeware kit goes to great lengths to thwart would-be pirates by introducing a hardware-based product activation scheme similar to what's found in Microsoft Windows. ... They've also pushed out multiple flavors of the package that vary in price depending on the capabilities it offers. Just as Windows users can choose between the lower-priced Windows 7 Starter or the more costly Windows 7 Business, bot masters have multiple options for Zeus.'"
Read more of this story at Slashdot.
WolverineOfLove writes "I'm recreating a 1980s abandonware game with copyrights that have been seemingly unused for the past 18 years. The situation is detailed further in a Slashdot journal entry I just wrote, but in short: Is it worth dealing with all the copyrights and paying money if I want to recreate an abandonware title as an open source game? I know there are legal implications to certain decisions I might make, but there is a real possibility that this game's copyright holder will do nothing with the rights, and I'd much prefer preserving it for others than letting it fade away."
Read more of this story at Slashdot.
Sometimes, we say things we just don't mean. Or, more accurately, that we really do mean, but we wish we hadn't said in public.
Over the past six weeks or so, I have been working on cleaning up the pluggable storage engine API in Drizzle. I’d like to describe some of this work and talk a bit about the next steps I’m taking in the coming months as we roll towards implementing Log Shipping in Drizzle.
First, how did it come about that I started working on the storage engine API?
Well, it really goes back to my work on Drizzle’s replication system. I had implemented a simple, fast, and extensible log which stored records of the data changes made to a server. Originally, the log was called the Command Log, because the Google Protobuffer messages it contained were called message::Commands. The API for implementing replication plugins was very simple and within a month or so of debuting the API, quite a few replication plugins had been built, including one replicating to Memcached, a prototype one replicating to Gearman, and a filtering replicator plugin.
In addition, Marcus Eriksson had created the RabbitReplication project which could replicate from Drizzle to other data stores, including Cassandra and Project Valdemort. However, Marcus did not actually implement any C/C++ plugins using the Drizzle replication API. Instead, RabbitReplication simply read the new Command Log, which due to it simply being a file full of Google Protobuffer messages, was quick and easy to read into memory using a variety of different programming languages. RabbitReplication is written in Java, and it was great to see other programming languages be able to read Drizzle’s replication log so easily. Marcus later coded up a C++ TransactionApplier plugin which replaces the Drizzle replication log and instead replicates the GPB messages directly to RabbitMQ.
And there, you’ll note that one of the plugins involved in Drizzle’s replication system is called TransactionApplier. It used to be called CommandApplier. That was because the GPB Command messages were individual row change events for the most part. However, I made a series of changes to the replication API and now the GPB messages sent through the APIs are of class message::Transaction. message::Transaction objects contain a transaction context, with information about the transaction’s start and end time, it’s transaction identifer, along with a series of message::Statement objects, each of which representing a part of the data changes that the SQL transaction made.
Thus, the Command Log now turned into the Transaction Log, and everywhere the term Command was used now was replaced with the terms Transaction and Statement (depending on whether you were talking about the entire Transaction or a piece of it). Log entries were now written at COMMIT to the Transaction Log and were not written if no COMMIT occurred1.
After finishing this work to make the transaction log write Transaction messages at commit time, I was keen to begin coding up the publisher and subscriber plugins which represent a node in the replication environment. However, Brian had asked me to delay working on other replication features and ensure that the replication API could support fully distributed transactions via the X/Open XA distributed transaction protocol. XA support had been removed from Drizzle when the MySQL binlog and original replication system was ripped out and needed some TLC. Fair enough, I said. So, off I went to work on XA.
As anyone who has worked on the MySQL source code or developed storage engines for MySQL knows, working with the MySQL pluggable storage engine API is sometimes not the easiest or most straightforward thing. I think the biggest problem with the MySQL storage engine API is that, due to understandable historical reasons, it’s an API that was designed with the MyISAM and HEAP storage engines in mind. Much of the transactional pieces of the API seem to be a bolted-on afterthought and can be very confusing to work with.
As an example, Paul McCullagh, developer of the transactional storage engine PBXT, recently emailed the mysql internals mailing list asking how the storage engine could tell when a SQL statement started and ended. You would think that such a seemingly basic functionality would have a simple answer. You’d be wrong. Monty Widenius answered like this:
Why not simply have a counter in your transaction object for how start_stmt – reset(); When this is 0 then you know stmnt ended.
In Maria we count number of calls to external_lock() and when the sum goes to 0 we know the transaction has ended.
To this, Mark Callaghan responded:
Why does the solution need to be so obscure?
Monty answered (emphasis mine):
Historic reasons.
MySQL never kept a count of which handlers are used by a transaction, only which tables.
So the original logic was that external_lock(lock/unlock) is called for each usage of the table, which is normally more than enough information for a handler to know when a statement starts/ends.
The one case this didn’t work was in the case someone does lock tables as then external_lock is not called per statement. It was to satisfy this case that we added a call to start_stmt() for each table.
It’s of course possible to change things so that start_stmt() / end_stmt() would be called once per used handler, but this would be yet another overhead for the upper level to do which the current handlers that tracks call to external_lock() doesn’t need.
Well, in Drizzle-land, we aren’t beholden to “historic reasons” 
So, after looking through the in-need-of-attention transaction processing code in the kernel, I decided that I would clean up the API so that storage engines did not have to jump through hoops to notify the kernel they participate in a transaction or just to figure out when a statement and a transaction started and ended.
The resulting changes to the API are quite dramatic I think, but I’ll leave it to the storage engine developers to tell me if the changes are good or not. The following is a summary of the changes to the storage engine API that I committed in the last few weeks.
The very first thing I did was to split the enormous base plugin class for a storage engine, plugin::StorageEngine, into two other subclasses containing transactional elements. plugin::TransactionalStorageEngine is now the base class for all storage engines which implement SQL transactions:
/** * A type of storage engine which supports SQL transactions. * * This class adds the SQL transactional API to the regular * storage engine. In other words, it adds support for the * following SQL statements: * * START TRANSACTION; * COMMIT; * ROLLBACK; * ROLLBACK TO SAVEPOINT; * SET SAVEPOINT; * RELEASE SAVEPOINT; */ class TransactionalStorageEngine :public StorageEngine { public: TransactionalStorageEngine(const std::string name_arg, const std::bitset<HTON_BIT_SIZE> &flags_arg= HTON_NO_FLAGS); virtual ~TransactionalStorageEngine(); ... private: void setTransactionReadWrite(Session& session); /* * Indicates to a storage engine the start of a * new SQL transaction. This is called ONLY in the following * scenarios: * * 1) An explicit BEGIN WORK/START TRANSACTION is called * 2) After an explicit COMMIT AND CHAIN is called * 3) After an explicit ROLLBACK AND RELEASE is called * 4) When in AUTOCOMMIT mode and directly before a new * SQL statement is started. */ virtual int doStartTransaction(Session *session, start_transaction_option_t options) { (void) session; (void) options; return 0; } /** * Implementing classes should override these to provide savepoint * functionality. */ virtual int doSetSavepoint(Session *session, NamedSavepoint &savepoint)= 0; virtual int doRollbackToSavepoint(Session *session, NamedSavepoint &savepoint)= 0; virtual int doReleaseSavepoint(Session *session, NamedSavepoint &savepoint)= 0; /** * Commits either the "statement transaction" or the "normal transaction". * * @param[in] The Session * @param[in] true if it's a real commit, that makes persistent changes * false if it's not in fact a commit but an end of the * statement that is part of the transaction. * @note * * 'normal_transaction' is also false in auto-commit mode where 'end of statement' * and 'real commit' mean the same event. */ virtual int doCommit(Session *session, bool normal_transaction)= 0; /** * Rolls back either the "statement transaction" or the "normal transaction". * * @param[in] The Session * @param[in] true if it's a real commit, that makes persistent changes * false if it's not in fact a commit but an end of the * statement that is part of the transaction. * @note * * 'normal_transaction' is also false in auto-commit mode where 'end of statement' * and 'real commit' mean the same event. */ virtual int doRollback(Session *session, bool normal_transaction)= 0; virtual int doReleaseTemporaryLatches(Session *session) { (void) session; return 0; } virtual int doStartConsistentSnapshot(Session *session) { (void) session; return 0; } };
As you can see, plugin::TransactionalStorageEngine inherits from plugin::StorageEngine and extends it with a series of private pure virtual methods that implement the SQL transaction parts of a query — doCommit(), doRollback(), etc. Implementing classes simply inherit from plugin::TransactionalStorageEngine and implement their internal transaction processing in these private methods.
In addition to the SQL transaction, however, is the concept of an XA transaction, which is for distributed transaction coordination. The XA protocol is a two-phase commit protocol because it implements a PREPARE step before a COMMIT occurs. This XA API is exposed via two other classes, plugin::XaResourceManager and plugin::XaStorageEngine. plugin::XaResourceManager derived classes implement the resource manager API of the XA protocol. plugin::XaStorageEngine is a storage engine subclass which, while also implementing SQL transactions, also implements XA transactions.
Here is the plugin::XaResourceManager class:
/** * An abstract interface class which exposes the participation * of implementing classes in distributed transactions in the XA protocol. */ class XaResourceManager { public: XaResourceManager() {} virtual ~XaResourceManager() {} ... private: /** * Does the COMMIT stage of the two-phase commit. */ virtual int doXaCommit(Session *session, bool normal_transaction)= 0; /** * Does the ROLLBACK stage of the two-phase commit. */ virtual int doXaRollback(Session *session, bool normal_transaction)= 0; /** * Does the PREPARE stage of the two-phase commit. */ virtual int doXaPrepare(Session *session, bool normal_transaction)= 0; /** * Rolls back a transaction identified by a XID. */ virtual int doXaRollbackXid(XID *xid)= 0; /** * Commits a transaction identified by a XID. */ virtual int doXaCommitXid(XID *xid)= 0; /** * Notifies the transaction manager of any transactions * which had been marked prepared but not committed at * crash time or that have been heurtistically completed * by the storage engine. * * @param[out] Reference to a vector of XIDs to add to * * @retval * Returns the number of transactions left to recover * for this engine. */ virtual int doXaRecover(XID * append_to, size_t len)= 0; };
and here is the plugin::XaStorageEngine class:
/** * A type of storage engine which supports distributed * transactions in the XA protocol. */ class XaStorageEngine :public TransactionalStorageEngine, public XaResourceManager { public: XaStorageEngine(const std::string name_arg, const std::bitset<HTON_BIT_SIZE> &flags_arg= HTON_NO_FLAGS); virtual ~XaStorageEngine(); ... };
Pretty clear. A plugin::XaStorageEngine inherits from both plugin::TransactionStorageEngine and plugin::XaResourceManager because it implements both SQL transactions and XA transactions. The InnobaseEngine is a plugin which inherits from plugin::XaStorageEngine because InnoDB supports SQL transactions as well as XA.
The second major change I made addressed the problem that Mark Callaghan noted in asking why finding out when a statement starts and ends was so obscure. I added two new methods to plugin::StorageEngine called doStartStatement() and doEndStatement(). The kernel now explicitly tells storage engines when a SQL statement starts and ends. This happens before any calls to Cursor::external_lock() happen, and there are no exception cases. In addition, the kernel now always tells transactional storage engines when a new SQL transaction is starting. It does this via an explicit call to plugin::TransactionalStorageEngine::doStartTransaction(). No exceptions, and yes, even for DDL operations.
What this means is that for a transactional storage engine, it no longer needs to “count the calls to Cursor::external_lock()” in order to know when a statement or transaction starts and ends. For a SQL transaction, this means that there is a clear code call path and there is no need for the storage engine to track whether the session is in AUTOCOMMIT mode or not. The kernel does all that work for the storage engine. Imagine a Session executes a single INSERT statement against an InnoDB table while in AUTOCOMMIT mode. This is what the call path looks like:
drizzled::Statement::Insert::execute()
|
-> drizzled::mysql_lock_tables()
|
-> drizzled::TransactionServices::registerResourceForTransaction()
|
-> drizzled::plugin::TransactionalStorageEngine::startTransaction()
|
-> InnobaseEngine::doStartTransaction()
|
-> drizzled::plugin::StorageEngine::startStatement()
|
-> InnobaseEngine::doStartStatement()
|
-> drizzled::plugin::StorageEngine::getCursor()
|
-> drizzled::Cursor::write_row()
|
-> InnobaseCursor::write_row()
|
-> drizzled::TransactionServices::autocommitOrRollback()
|
-> drizzled::plugin::TransactionStorageEngine::commit()
|
-> InnobaseEngine::doCommit()
I think this will come as a welcome change to storage engine developers working with Drizzle.
There was an interesting comment in the original documentation for the transaction processing code. It read:
Roles and responsibilities
————————–The server has no way to know that an engine participates in
the statement and a transaction has been started
in it unless the engine says so. Thus, in order to be
a part of a transaction, the engine must “register” itself.
This is done by invoking trans_register_ha() server call.
Normally the engine registers itself whenever handler::external_lock()
is called. trans_register_ha() can be invoked many times: if
an engine is already registered, the call does nothing.
In case autocommit is not set, the engine must register itself
twice — both in the statement list and in the normal transaction
list.
That comment, and I’ve read it dozens of times, always seemed strange to me. I mean, does the server really not know that an engine participates in a statement or transaction unless the engine tells it? Of course not.
So, I removed the need for a storage engine to “register itself” with the kernel. Now, the transaction manager inside the Drizzle kernel (implemented in the TransactionServices component) automatically monitors which engines are participating in an SQL transaction and the engine doesn’t need to do anything to register itself.
In addition, due to the break-up of the plugin::StorageEngine class and the XA API into plugin::XaResourceManager, Drizzle’s transaction manager can now coordinate XA transactions from plugins other than storage engines. Yep, that’s right. Any plugin which implements plugin::XaResourceManager can participate in an XA transaction and Drizzle will act as the transaction manager. What’s the first plugin that will do this? Drizzle’s transaction log. The transaction log isn’t a storage engine, but it is able to participate in an XA transaction, so it will implement plugin::XaResourceManager but not plugin::StorageEngine.
So, that “yet another overhead” Monty talked about in the quote above? There wasn’t any noticeable impact in performance or scalability at all. So much for optimize-first coding.
The next thing I’m working on is removing the notion of the “statement transaction”, which is also a historical by-product, this time because of BerkeleyDB. Gee, I’ve got a lot of work ahead of me…
[1] Actually, there is a way that a transaction that was rolled back can get written to the transaction log. For bulk operations, the server can cut a Transaction message into multiple segments, and if the SQL transaction is rolled back, a special RollbackStatement message is written to the transaction log.
Apple's recent purge of sex-tinged iPhone apps, combined with its lesser-known ban of cookie-cutter apps, signifies the company's new focus on quality,rather than quantity, in its App Store.
The Wall Street Journal reports that Apple's Chief Operating Officer, Tim Cook, received a bonus worth $22 million for his work filling in for Steve Jobs' during the CEO's 2009 medical leave.
Gabe Newell used his awards speech to speak about his thoughts on the industry... including his thoughts on DRM.
The iPad officially became available for pre-order earlier today and while plenty of you are still debating about whether or not to pre-order, it appears as if the device might already sold 51,000 units in just two hours via pre-order. Wow.
CuteSteveJobs writes "Apple has been dealt a severe blow having been told that it no longer has a monopoly on the letter 'i' for product naming. IP Australia, the government body that oversees trademark applications, rejected Apples' complaint against a company selling 'DOPi' laptop bags. Last year Australian computer company Macpro Computers claimed that after 26 years of flying its own Macpro brand that Apple was 'trying to burn us out' with legal fees. This was after Apple released its own Macpro line 3½ years ago. Apple lost that complaint, but is appealing. Last year Apple went after supermarket Woolworths complaining their new logo which featured a 'W' fashioned into the shape of an apple. (Woolworths sells real apples.)"
Read more of this story at Slashdot.
I didn't blog for almost three months! It has been a hectic few months, I've had so much on my plate I could hardly get anything done. So I spent some time this week unsubscribing from tens of mailing lists, and even more website newsletters, trying to bring my incoming emails considerably down so I can actually get work done. I think I'm there now. I'm still on all the core lists relevant to my work (font & text related lists; and haven't unsub'ed from foundation-list yet) and hope to do a better job pushing harfbuzz as well as maintaining pango, fontconfig, and vte. Anyway, it's easy enough to find my email. Feel free to send me mail or CC me on discussion I should be reading. And I try to do a better job of posting what I'm up to.
