PIFTS.EXE: Symantec finally own up

So, the world can rest easy in their beds. A message tucked away on Symantec's forums-- the same forums from which all communication about the issue was previously banned-- in which they have finally owned up to what happened:

• they released a patch to do some boring things that any old patch might have done
• but they released the patch unsigned, causing it to hit the firewall when it otherwise wouldn't have done
• because some of the posts on the Symantec forum were judged to be abusive, all posts were pulled down.

Whilst this seems to be an astonishing example of customer relations, and has brought the world's attention to the kind of behaviour that such patches may be conducting on a routine basis, it does at least appear that the Feds are not about to plunder our computers for illicit chocolate chip cookie recipes. We were spared... this time.

(And yes, I did back up my recipe collection... just in case.)

What is the mysterious PIFTS.EXE?

Update 10/03/09 11:50am Luckily, it does appear that PIFTS.EXE is just a storm in a teacup. Symantec still appear to be saying about as much as the Queen did after Diana died.

So after a mysterious PIFTS.EXE program hits the Kaspersky firewall asking to connect out from one of our machines, I hit the Internet to find that nobody knows, but the world is wondering. According to Google Trends, it has been hovering between the 15th and 25th most frequent search for the last couple of hours. Various theories about PIFTS.EXE appear to be emerging: was it some component of Norton Antivirus that went wrong? Is it some mad terrorist plot to wipe the Internet off the face of the earth and thus prevent people from finding out about why Lil' Kim went to jail?

Update 10/03/09 Reading things so far, and just possibly maybe having had a quick look at a disassembly of the exe in IDAPro, the consensus seems to be that the file is essentially harmless, but was an attempt by Symantec to gather some statistics from users' machines about installed antivirus components. A user posting to Reddit also suggests that a code review of PIFTS.EXE does not reveal anything too nafarious, and that automated code analysers such as ThreatExpert don't pull anything up either (then again, a "well written" virus wouldn't do, would it?).

Sometimes in cases like this, it's not so much whether there is anything wrong but whether there appears to be. Pulling down all forum posts about the file when there is clear user anxiety without then making an official statement doesn't making it look as though everything's hunky-dory...

Java Certification Tips

A new page of Java Certification Tips gives a "cribsheet" of some of the "syntactic niggles" and other commonly overlooked features of Java that could trip up an otherwise moderately experienced programmer when taking the Sun Certified Java Programmer (SCJP) exam. Did you know, for example that:

• this is invalid Java syntax?
  float f = 2.5;
• goto is a Java keyword?
• this line does not assign the value 13 to the variable?:
  int i = 013;

See the page of tips for more information on these and other niggles.

Regular expression tutorial: new example

The Java regular expression tutorial section has been updated with a new example of using regular expressions. In this example, we look at how to perform what is sometimes referred to as HTML scraping: pulling out data from an HTML page (or indeed XML document).

The example is located here: HTML scraping with Java regular expressions. As explained in this tutorial, regular expressions are a good candidate for data scraping because they are flexible. Various libraries exist to parse an HTML or XML document and return an object representation of that document. But such libraries are often "too fussy": many if not most web pages actually do not conform strictly to HTML standards. Similarly, many XML parsing libraries are too fussy for real-life RSS feeds, which are often malformed, strictly speaking. Using regular expressions cuts through the fuss.

When should you scrape web pages?

Note that the article focusses on how technically to scrape HTML pages in Java. It doesn't deal with the "political" issue of whether the site in question wants its content scraped in the first place. In general, it is good practice to do the following:

• find out if he web site in question has an API to provide the data in a more convenient format (and in a format that it would prefer to provide it to you in!)
• be open about what you're doing: if the web site administrator things you're trying to "hide" something, they may block your IP address
• respect server resources: if you are retrieving multiple pages, consider putting a thread sleep between fetches
  

Updates to the Spanish-English glossary

Various new entries have been added to the Javamex site's Spanish-English glossary of computing terms. For those not familiar with the glossary, it contains English translations of various Spanish computing terms, covering various IT topics such as programming, networking, the Internet, software and hardware, GUIs etc.

Information on memory usage of objects

The section on Java memory usage now contains the following additional articles:

• information on how to calculate the memory usage of a Java object in general, considering the memory used for "housekeeping" by the JVM
• calculating the memory usage of Strings, which can often the type of object to use up the biggest proportion of space in a Java application: this section actually considers the memory use of string-related objects such as StringBuffers and StringBuilders

A section on reducing the memory taken up by Strings looks at string canonicalisation, a fairly standard approach (but one which requires certain caveats), plus introduces the example of a CompactCharSequence class, that stores strings as 1 byte per character, thus taking up around half othe memory taken up by a regular Java String (at the expense of not supporting Unicode).

Comments on these articles welcome as usual.

Beta: Classmexer agent

The beta version of a simple instrumentation tool is available for download from the Javamex site. The Classmexer jar provides various calls for querying the memory usage of Java objects. Via the provided MemoryUtil class.deepMemoryUsageOf() method, it is possible to get an estimate from the JVM of the number of bytes taken up by an object and its "subobjects" (objects referred to by a non-public reference, or by references with other visibility criteria). The memory usage of subobjects is combined recursively (so subobjects of subobjects are considered etc), but without counting the same object more than once.

A variant of the call is also provided which gives the total memory usage of several objects at a time, without counting as duplicates objects referenced by more than one of the objects.