Handling large data files efficiently with Java

Grok Headline matches for Handling large data files efficiently with Java

O'Reilly: Handling Missing
Persons....er, Files

O'Reilly: Handling Missing
Persons....er, Files 02/14/2003 04:37 PM

Pinging really large files

Pinging really large files 06/19/2004 06:25 PM
As spotted by Kottke, the combined denizens of the MT Support boards have found that older versions of MT with Trackback-Autodiscovery choke when you link to large files. This explains a great deal: nasty gotcha-bug too. Worth bearing in mind...

Serving Large PDF Files

Serving Large PDF Files 04/06/2005 03:22 PM

If you have large PDF files at your site, you might want to read this Microsoft technote explaining how IE handles content types associated with [Netscape] plug-ins (i.e. Acrobat) and programmer-level workaround Mat t Raible.  I suspect IE's weird triple request was to support some misbehaving yet popular plugins.

I am not sure if popular web servers like Apache and IIS comes with built-in filters to address this problem.  If not, then this opens up another golden shareware moment.

Event Handling In Java

Event Handling In Java 10/28/2003 11:06 PM
In life, you encounter events that force you to suspend other activities and respond to them immediately. In Java, events represent all activity that goes on between the user and the application. Java's Abstract Windowing Toolkit (AWT) communicates these actions to the programs using events.

Exception handling in Java

Exception handling in Java 05/27/2004 12:16 AM
No, this isn't yet another rant on checked vs non-checked exceptions. It's just a link to an excellent article on exception handling by Heinz Kabutz. Heinz drew me into reading the article with the very first sentence (which I completely agree with): In my experience, all Java projects contain poorly written exception handling code. He then shows how exception handling... (176 words)

Event Handling In Java Part II

Event Handling In Java Part II 10/28/2003 11:06 PM
With the skills that you have developed so far from Part I of the tutorial, you can design a graphical user interface with beauty and easy. Let us refresh ourselves before we proceed, Events are method calls that Javas windowing system performs whenever any element of a user interface is manipulated.

[ GLSA 200501-02 ] a2ps: Insecure
temporary files handling

[ GLSA 200501-02 ] a2ps: Insecure
temporary files handling 01/05/2005 04:04 PM
Thierry Carrez (Jan 04 2005)

Despoolr Directory Has a Large Number of
Files

Despoolr Directory Has a Large Number of
Files 12/19/2004 03:26 PM

Efficient Techniques for Modifying Large
XML Files

Efficient Techniques for Modifying Large
XML Files 04/22/2004 01:20 AM
As XML has become popular as a representation format for large sources of information, developers have begun to have problems with editing large XML files. This is especially true for applications that process large log files and need to constantly append information to these files. The most straightforward way to edit an XML file is to load it into an XmlDocument, modify the document in memory and then save it back to disk. However, doing so means that the entire XML document has to be loaded in memory, which may be infeasible depending on the size of the document and the memory requirements of the application.

Open 'in use' files during large copy
operations

Open 'in use' files during large copy
operations 08/27/2004 01:38 PM
When copying a large file from a network volume mounted in the Finder, the destination file cannot be opened (while the copy is in progress) by double-clicking it in the Finder or via navigation dialogs in Mac OS X applicatio...

Handling Arbitrary RSS Data with
NewsGator

Handling Arbitrary RSS Data with
NewsGator 01/24/2004 10:36 PM

RSS extensions: The inventor of NewsGator shows us some of the really cool things you can do with it.

...you can define whatever extension you like, and NewsGator will be able to use it in one way of another. Let's explore how this works, and what you can do...you can take any extension data and map it to a column in Outlook...NewsGator 2.0's rendering engine now uses XSLT, and you can customize the transforms used to render the items. What's more, any custom data is available to these transforms...so you could have a custom rendering based on this data by building a new stylesheet.

This could be huge for corporate applications — RSS could be a generalized reporting stream on anything, including any statistics necessary. Using Outlook VBA, you can even trigger client-side programs when posts are received, so there's really no limit on where you could go with this.

Click here to comment on this entry

HotFix Watch: Large MIF hardware
inventory files process slowly

HotFix Watch: Large MIF hardware
inventory files process slowly 04/29/2004 05:53 PM

Consumers Delete Large Numbers Of
Digital Music Files From PC's

Consumers Delete Large Numbers Of
Digital Music Files From PC's 11/06/2003 07:38 AM

GMDH - Group Method of Data Handling

GMDH - Group Method of Data Handling 09/04/2004 06:23 AM
GMDH - Group Method of Data Handling
http://come.to/GMDH

Group Method of Data Handling was applied in a great variety of areas for data mining and knowledge discovery, forecasting and systems modeling, optimization and pattern recognition. Inductive GMDH algorithms give possibility to find automatically interrelations in data, to select optimal structure of model or network and to increase the accuracy of existing algorithms. This original self-organizing approach is substantially different from deductive methods used commonly for modeling. It has inductive nature - it finds the best solution by sorting-out of possible variants. By sorting of different solutions GMDH networks aims to minimize the influence of the author on the results of modeling. Computer itself finds the structure of the model and the laws which act in the system. Group Method of Data Handling is a set of several algorithms for different problems solution. It consists of parametric, clusterization, analogues complexing, rebinarization and probability algorithms. This has been added to Data Mining Resources Subject Tracer™ Information Blog and Knowledge Discovery Subject Tracer™ Information Blog.

Book review: Java & XML Data Binding
(Java Pro)

Book review: Java & XML Data Binding
(Java Pro) 08/26/2002 08:32 AM

ObjectStore Set to Launch One-Stop Shop
for RFID Data Handling

ObjectStore Set to Launch One-Stop Shop
for RFID Data Handling 09/20/2004 01:18 AM
ObjectStore is releasing what it hopes will become a one-stop-shop for businesses to manage the swamp of data coming from RFID applications.

Mohler, Nixon & Williams Selects Data
Domain Recovery Appliance to Manage
Large Volumes of Backup Data; New
Appliance Proves Key to Company's
Disaster Recovery/Business Continuation
Strategy

Mohler, Nixon & Williams Selects Data
Domain Recovery Appliance to Manage
Large Volumes of Backup Data; New
Appliance Proves Key to Company's
Disaster Recovery/Business Continuation
Strategy 07/29/2004 03:03 AM
Mohler, Nixon & Williams has recently selected the Data Domain Recovery Appliance to manage large volumes of backup data. The new recovery appliance will be key to the public accounting firm'sDisaster Recovery/Business Continuation Strategy. The company feels the DD200 has shortened their backup window by 50 percent. [PRWEB Jul 29, 2004]

W3C Web Services Working Groups Release
New Drafts on Handling of Binary Data

W3C Web Services Working Groups Release
New Drafts on Handling of Binary Data 06/09/2004 03:56 PM
XMLMania.com Jun 9 2004 7:08PM GMT

55 percent of large U.S. businesses will
deploy a wireless wide-area data
solution by mid-2006

55 percent of large U.S. businesses will
deploy a wireless wide-area data
solution by mid-2006 09/15/2004 03:57 AM
[PRWEB Sep 15, 2004]

Spinks Technologies migrates large
publicly traded pharmaceutical
corporation into production on Oracle
and Electronic Data Interchange.

Spinks Technologies migrates large
publicly traded pharmaceutical
corporation into production on Oracle
and Electronic Data Interchange. 09/24/2004 03:13 AM
Gerard Spinks, CEO of Spinks Technologies, announces that his firm has successfully installed and implemented Oracle and EDI to power large pharmaceutical company supply chain integration efforts with the largest retailers in the United States. [PRWEB Sep 24, 2004]

Search the Web More Efficiently

Search the Web More Efficiently 12/03/2003 07:30 PM
Search the Web More Efficiently
http://www.llr x.com/features/efficientsearch.htm

Search engine and website design expert Daniel Bazac identifies search tools, features and strategies that will contribute to your successful research efforts on the web.

NetNewsWire updated; browse the Web more
efficiently

NetNewsWire updated; browse the Web more
efficiently 03/19/2003 10:25 PM
The headlines pane can show dates, subjects, and creators when available. The Find command allows you to search through current news. It's scriptable via AppleScript and other OSA languages. There's a Scripts menu where you can add your own scripts.

Efficiently Updating Web Sites on
Clusters

Efficiently Updating Web Sites on
Clusters 01/15/2003 12:51 PM
At the time, our web site ran on a 2.2.x Linux distribution on a dual Dell 2450. Its performance was rock solid, but we were uneasy relying on a single machine for our entire business. We determined that the answer was to replace the single server with a cluster. For less than the cost of the Dell, we built a cluster of four 1U one-processor machines. The cluster's performance was excellent. By stripping down Apache, we could support at least 400 downloads over HTTP and still have a responsive site. This left one problem: we needed to be able to update the site often without affecting the performance. -- M. David Minnigerode

"zeldman.pur"

Use LimeWire to transfer data files to a
selected user

Use LimeWire to transfer data files to a
selected user 04/07/2005 09:19 AM
As a photographer, I often have the need to transfer large documents (mostly images) to my clients. I had considered an FTP site, but for economic reasons, I opted for a .Mac account (iDisk) to partially fulfill my business n...

Data Wizard for Java 1.0.1

Data Wizard for Java 1.0.1 11/05/2003 08:16 PM
A tool for building a simple graphical user interface.

JiBX - XML Data Binding for Java

JiBX - XML Data Binding for Java 04/30/2004 11:58 AM
Beta 3b release

WKB4J: A reader for WKB data in Java

WKB4J: A reader for WKB data in Java 12/13/2003 08:42 PM
PostGIS 0.8, Postgresql 7.4

Recover My Files Data Recovery Software
Recovers More Than 100 File Types

Recover My Files Data Recovery Software
Recovers More Than 100 File Types 04/07/2005 02:49 AM
GetData Software Development Company's Recover My Files will be launched into the US retail market this month. [PRWEB Apr 7, 2005]

TriActive Java Data Objects 2.0 (Stable)

TriActive Java Data Objects 2.0 (Stable) 11/11/2003 06:53 PM
An implementation of Sun's JDO specification (JSR-12).

Sun seeks data persistence model for
Java

Sun seeks data persistence model for
Java 09/24/2004 08:04 PM
Sun Microsystems is looking to make life easier for Java developers by seeking a common ground on a data persistence model.

TriActive Java Data Objects (TJDO)

TriActive Java Data Objects (TJDO) 11/11/2003 04:27 PM
TJDO 2.0 Released

AdventNet Releases A Freeware Version of
OpManager To Efficiently Manage Small
Networks

AdventNet Releases A Freeware Version of
OpManager To Efficiently Manage Small
Networks 08/03/2004 02:09 AM
AdventNet today announced the release of a freeware version of its web-based network, systems and application management software, ManageEngine OpManager 5. The free version is suited for small business IT administrators and can be used for monitoring 10 critical devices. [PRWEB Aug 3, 2004]

Avoid Java-XML data binding headaches
(ZDNet UK)

Avoid Java-XML data binding headaches
(ZDNet UK) 09/18/2002 03:29 PM

Dharma Provides Data Access Solution for
Java Applications

Dharma Provides Data Access Solution for
Java Applications 03/17/2005 03:36 AM
Dharma Systems, Inc., releases Dharma SDK for Java. [PRWEB Mar 16, 2005]

Store and retrieve XML data with custom
Java classes in Sybase

Store and retrieve XML data with custom
Java classes in Sybase 07/11/2002 12:57 AM
CNET Jul 11 2002 0:04AM ET

Tangosol Upgrades Data Management Tool
for Java Apps

Tangosol Upgrades Data Management Tool
for Java Apps 07/27/2004 01:06 PM
Coherence 2.4 adds support for IBM's WebSphere as well as a new caching model.

CodeFutures Updates FireStorm/DAO Java
Code Generator for Data Persistence

CodeFutures Updates FireStorm/DAO Java
Code Generator for Data Persistence 03/14/2005 05:57 PM
CodeFutures has announced the General Availability of Release 2.4 of its award-winning FireStorm/DAO tool that makes software developers more productive by generating the Java code for accessing databases. The benefits provided by CodeFutures' code generation approach are higher developer productivity, better software quality, and lower maintenance costs. [PRWEB Mar 10, 2005]

Advanced Error Handling: Writing an
Error Handling Class

Advanced Error Handling: Writing an
Error Handling Class 11/10/2003 11:25 PM
If you're tired of the default error handler and want to have complete control over default error messages, you should write your own error handling class. Writing your own handler will enable you to change the way php handles your error messages, and allows you to create your own error types. With this class you will be able to send error messages to a log file, or send error reports via email.

OpenOffice World-Readable Temporary
Files Disclose Files to Local Users

OpenOffice World-Readable Temporary
Files Disclose Files to Local Users 09/10/2004 01:05 PM
Jérôme (Sep 10 2004)
Grok Description matches for Handling large data files efficiently with Java
GrokA matches for Handling large data files efficiently with Java

Loki Torrent - Torrent Search, Torrent
Download, You name it, we've got it.

Loki Torrent - Torrent Search, Torrent
Download, You name it, we've got it. 12/30/2004 11:53 AM
fights back .. lokittorrent .. Loki Torrent

lokitorrent.com
track this site | 5 links

"Loki Torrent - Torrent Search, Torrent
Download, You name it..."

"Loki Torrent - Torrent Search, Torrent
Download, You name it..." 12/31/2004 10:23 AM

ALL MULTIPLE P2P NETWORK FILE SHARING
SOFTWARE ARE NOT THE SAME: TrustyFiles
2.2 update adds Bit Torrent access and
delivers the fastest and most results
and download sources with 100% native
code.

ALL MULTIPLE P2P NETWORK FILE SHARING
SOFTWARE ARE NOT THE SAME: TrustyFiles
2.2 update adds Bit Torrent access and
delivers the fastest and most results
and download sources with 100% native
code. 06/07/2004 02:37 AM
RazorPop, Inc. announced the release of TrustyFiles 2.2 Personal File Sharing software at http://www.TrustyFiles.com. The performance-driven update cements TrustyFile’s position as the leader in Multi-P2P network software. TrustyFiles 2.2 features 100% native code and adds Bit Torrent network support. TrustyFiles continues to be FREE with NO spyware and NO additional bundled software. [PRWEB Jun 7, 2004]

"bitoogle :: the bit torrent file search
engine (bittorrent)"

"bitoogle :: the bit torrent file search
engine (bittorrent)" 07/13/2004 08:44 PM

bitoogle :: the bit torrent file search
engine (bittorrent)

bitoogle :: the bit torrent file search
engine (bittorrent) 07/14/2004 01:18 AM
bitoogle :: the bit torrent file search engine .. bitoogle

bitoogle.com
track this site | 4 links

My first torrent

My first torrent 07/30/2004 03:00 AM

Thanks to Jim and Ado for setting up the BitTorrent tracker. Here is a torrent for Lawrence Lessig's Free Culture talk in Helsinki that I blogged about earlier.

UPDATE: Please standby. It doesn't seem to be working.

Comment - TrackBack

Torrent Zip

Torrent Zip 03/31/2005 11:44 PM
The project is live!

Torrent 0.61

Torrent 0.61 01/27/2004 02:58 PM
An arcade game with colored tiles.

It's the torrent, stupid

It's the torrent, stupid 12/22/2004 01:29 AM
Xeni Jardin: Mark Pesce rants about the recent shutdowns of BitTorrent supersites Suprnova.org and TorrentBits.com.

Hey, Hollywood! Can you feel the future slipping through your fingers? Do you understand how badly you've screwed up? You took a perfectly serviceable situation - a nice, centralized system for the distribution of media, and, through your own greed and shortsightedness, are giving birth to a system of digital distribution that you'll never, ever be able to defeat. In your avarice and arrogance you ignored the obvious: you should have cut a deal with SuprNova.org. In partnership you could have found a way to manage the disruptive change that's already well underway. Instead, you have repeated the mistakes made by the recording industry, chapter and verse. And thus you have spelled your own doom.

It's said that the best sequels are just like the original, only bigger and louder. Ladies and gentlemen, prepare yourselves for one hell of a crash. This baby is now fully out of control.

Link (via waxy)

Following up on Torrent Shutdowns

Following up on Torrent Shutdowns 12/22/2004 01:40 AM
Slashdot Dec 21 2004 6:33PM GMT

Bit Torrent : An Analysis

Bit Torrent : An Analysis 12/19/2004 03:10 PM
Hardy news site, The Register, recently published a detailed analysis of the file sharing protocol Bit Torrent. Bit Torrent has received attention in the main stream news after reports that it was carrying as much as 50% of all peer 2 peer (p2p) traffic, which in tern amounted to a massive 30% of all the traffic on the internet. The paper, by Dr. Johan Pouwelse, examines the protocol and looks especially at one of the largest bit-torrent hubs, Suprnova.org. He examines how just 20 moderators solve the problem of fake files, something that plagues the traditional file sharing networks like Kazaa.

Dr Powelse notes that the major problems facing hubs like suprnova are fakes and maintaining hub availability. The availability of files on bit torrent is based on a centralised system; without it, the network fails as users cannot access the trackers. Decentralising bit torrent has already begun - Suprnova have started a project called "Exeem" which apparently has 5,000 beta testers trialling it, and has an ultimate aim of taking the best of Kazaa (a decentralised network) and merging it with Bit Torrent. Decentralisation removes the issue of poor availability at the tracker end, yet0 it also provides more scope for fake files and a reduction in data integrity at the user end.

The paper concludes that bit-torrent needs to evolve to create incentives to users to seed files. Bit-torrent as a protocol is a system that’s here to stay; it enjoys more and more usage from more main stream content providers. Yes, there is a lot of illegitimate use of the protocol, but unlike Kazaa, these users should not be allowed to over shadow the usefulness to legitimate users of the bit torrent protocol.

[Update] Since this article was published, Suprnova has shutdown as a hub for torrents. Although this cannot be confirmed, the shutdown is very likely related to legal action from the Hollywood against tracker websites; earlier in the week many other sites were taken down. The effectiveness of the takedowns could be massive; the paper below notes that when on the Suprnova mirrors went offline during their monitoring period, they saw a massive reduction in the number of users downloading files through the site.

Download: The Paper (pdf) | The Register

Read full story...

Xcode .torrent

Xcode .torrent 08/08/2004 02:13 AM
Apple just released an update to is Xcode development tools, but Apple's content distribution network is slow and poky, and as Danny notes, it "won't let you resume downloads using wget -c." So here's a .torrent for Xcode. Link (via Oblomovka)

ShiftyGames Torrent 0.8.2

ShiftyGames Torrent 0.8.2 05/05/2004 10:52 PM
An arcade game with colored tiles.

Bit Torrent question

Bit Torrent question 04/09/2004 10:30 PM
Bit Torrent and the ability to download everything in one click (is this the end of Direct TV, Tivo and the music business?!).

Used BitTorrent a little bit when it first came out and was a bit underwhelmed. It didn’t work, there weren’t a lot of places to find files, etc.

I decided to take another look at it when a designer friend of mine was telling me that he has the latest version of every single piece of design software on his Mac compliments of bit torrent (yes, I know it’s wrong… not the point I’m trying to make, the point is coming :-).

Part I: I installed bit torrent and immediately noticed an amazing new trend (prob. not new to all of you) of people posting dozens of albums in one RAR file for download. Huge file sizes in the 500 to 4,000 meg size range. The last season of seven seasons of Southpark, every Nirvanna album and here is another file with every Howard Stern radio show from March in one file.

In one click you grab one really well organized, clean and deep sets of files—scary.

Part II: A couple of month ago I got the Gateway Connected DVD player. For $195 it connects via WiFi to my desktop and I can hit the My Music or My Videos button on the remote control and pull up those directories on my hard drive (in the other room).

Part III: Today I moved into my new apartment in Santa Monica and was faced with the standard $100 month cable/dish bill and I’m thinking “dang, I only watch less then a half dozen TV shows and they are all here on bit torrent… maybe I should save the $1,200 a year and just download the shows and watch them via my Gateway Connected DVD player?”

The Point/Question: How soon before you’ll be able-with one click-download every prime-time TV show or last year’s top 500 CDs in one click?!

(Note: This is not a trick question, I have yet to find a file containing that much content—however, I did find a file with last weeks top 100 singles that someone put together in one nice package).

[The Digital Music Weblog]

Sri Lankan hip-hop mix: torrent

Sri Lankan hip-hop mix: torrent 03/17/2005 03:55 AM
Xeni Jardin: Boing Boing reader Lucas Emery says,

Your big article on M.I.A over the weekend reminded me that I had downloaded a mix mp3 shortly after the Tsunami disaster comprised exclusivly of Sri Lankan hip-hop. I can't remember where I originally found the mix (boomselection, maybe?) so I just made a .torrent. 58.4 Meg mp3 mix by Dr. Auratheft.

Link

Previously: M.I.A. is, well, MIA; and MIA for intergalactic overlord

Torrentocracy = RSS + Bit Torrent + Your
TV

Torrentocracy = RSS + Bit Torrent + Your
TV 06/21/2004 07:41 AM

Battle Torrent

Battle Torrent 08/11/2004 09:45 AM

Thanks to Dave over at Scripting News for the link. The already easy process of downloading files via BitTorrent has just gotten easier. [Downhill Battle]

CC Torrent Hosting

CC Torrent Hosting 12/17/2004 06:33 PM

Torrentocracy has announced a free BitTorrent hosting service for Creative Commons licensed content: Prodigem.

Download one of the beta torrents currently available. Send an email to Torrentocracy creator Gary Lerhaupt to request an upload account.

Update: Download all of the Duke Law School Arts Project Moving Image Contest finalists via one torrent at prodigem.

great piano rendition of the Mario theme

great piano rendition of the Mario theme 02/13/2004 02:41 PM
we've all heard the song, but somehow this rendition just brings it all back to life

Torrent of video from DV Guide

Torrent of video from DV Guide 08/30/2004 02:55 AM

dv.open4all.info/bblog/torrent_files/20040828_kinberg.mov.torrenttrack this site | 3 links

Grokster briefs torrent

Grokster briefs torrent 03/26/2005 05:13 AM
Cory Doctorow: Thad sez, "This is a torrent of all of the briefs submitted re: MGM v. Grokster, in the zip format provided on the U.S. Copyright Office site." Torrent Link for 74 briefs in 20.7MB

Torrent Site Status

Torrent Site Status 01/07/2005 04:15 AM
Don’t download too much pr0n .. Torrent Site Status

orbdesign.net/bt
track this site | 3 links

SP2 Bit Torrent Legal Challenge

SP2 Bit Torrent Legal Challenge 08/11/2004 05:20 PM

Download the Windows XP Service Pack 2: The guys who were doing Microsoft a favor by pushing Service Pack 2 via Bit Torrent got slapped down by Redmond.

Microsoft sent DMCA takedown notices to our two webhosts, one of which was just linking to a torrent file on another server. We've stood up to these kinds of legal threats before (see the Grey Tuesday protests), but we decided not to bother this time, because we started this site primarily as a demonstration and to that end it's already been a huge success.

Click here to comment on this entry

Better Than Bit Torrent, For Internet2
Users?

Better Than Bit Torrent, For Internet2
Users? 11/18/2003 07:54 PM

"Torrent Link for 74 briefs in 20.7MB"

"Torrent Link for 74 briefs in 20.7MB" 03/27/2005 10:28 AM

Comrade - Bit Torrent Client

Comrade - Bit Torrent Client 06/24/2004 12:03 AM
Working.

Microsoft builds a better Bit Torrent

Microsoft builds a better Bit Torrent 06/17/2005 03:18 PM
Researchers at Microsoft's computer science lab in Cambridge have developed a peer-to-peer filesharing system that they say overcomes the scheduling problems associated with existing distribution protocols such as Bit Torrent.

The researchers claim download times are between 20-30 per cent faster, using their network coding approach, than on systems that only code at the server, and between 200 and 300 per cent faster than distributing un-encoded information.

View: Full Article @ The Register
View: Avalanche Whitepaper

Read full story...

Outfoxed interviews .torrent for
remixing

Outfoxed interviews .torrent for
remixing 09/15/2004 11:36 AM
Cory Doctorow: The interviews from the awesome anti-Fox documentary Outfoxed have been released under a Creative Commons license, for you to remix. Here's the .torrent: Torrent Link (via Lessig)

Defense fund for Bit Torrent indexer

Defense fund for Bit Torrent indexer 12/30/2004 02:45 AM
Cory Doctorow: LokiTorrent is a BitTorrent indexing site -- like the lamented Suprnova -- that has been threatened with legal action by the MPAA for telling people where to download torrent files that allow them to download video and other large data-objects. Unlike some of the other Torrent indexers that shut down last week, LokiTorrent is mounting a legal defense. They're trying to raise a legal defense fund of $30,000, and they've made $11,500 in the first 12 hours. Link (via /.)

BeeTV - automatic torrent downloader

BeeTV - automatic torrent downloader 04/15/2005 10:02 AM
Prototype version released

Handling large data files efficiently with Java