Harvester Design

[conseo]

Hi,

> 
> http://whiletaker.homeip.net/votorola/harvester/javadoc/votorola/a/diff/harv
> est/package-summary.html#package_description (i) The methods on the kicker
> (listen and trigger) should proabably be renamed "register" and "raise" for
> consistency.

Done.

> 
> (ii) Can the table at bottom link to the named components using
> {@link} or something?  That'll make it easier to keep up to date,
> because Javadoc will warn you when names change, etc.

Done.

> 
> (iii) "Cache" is incorrect in the table.

Fixed.

> 
> > >      b) Kicker
> 
> http://whiletaker.homeip.net/votorola/harvester/javadoc/votorola/a/diff/harv
> est/Kicker.html#register%28votorola.a.diff.harvest.Kicker.EventHandler,%20ja
> va.lang.String%29 (i) I guess archiveFormat should be archiveDesign and
> should link to http://zelea.com/w/Property:Archive_design for the meaning. 
> I guess the format should be the simple name of the design without
> namespaces etc, e.g. just "Pipermail" for
> http://zelea.com/w/Stuff:Pipermail
> 
> (ii) The forum pages (and forum design, etc) will eventually be in a
> dedicated wiki, which usually won't be the admin's local pollwiki.
> After your code is running, we'll teach WikiCache how to work with
> multiple wikis.  We'll add a config item pointing to the forum wiki.
> (Please add this design note to your code and back-link to this post.)

Done.

> 
> http://whiletaker.homeip.net/votorola/harvester/javadoc/votorola/a/diff/harv
> est/Kicker.Event.html (iii) A suggestion: I would break it out as a separate
> class because it's a fairly important thing.  I would call it simply Kick,
> because a kick is an event (!) by definition.

Done.
> 
> (iv) I think the documented purpose of the Kick (or Kicker.Event)
> should be more specific.  Maybe: "An event signalling that a forum's
> local cache of messages might need to be refreshed.  The typical kick
> receiver is a harvester.  On receiving a kick, the harvester schedules
> a harvest job in which newly added messages are read from the forum
> archive and stored in the cache."  Something like that.

Done.
> 
> (v) I think the forum method should point to
> http://zelea.com/w/Concept:Forum for it's documentation.  It should
> return the simple name of the forum page without namespaces etc,
> e.g. "Metagovernment" for: http://zelea.com/w/Stuff:Metagovernment

Done.

> 
> http://whiletaker.homeip.net/votorola/harvester/javadoc/votorola/a/diff/harv
> est/Kicker.EventHandler.html (vi) Again, I would make it a top level class. 
> Maybe "KickReceiver", with a method called KickReceiver#receive(Kick)? 
> Just a suggestion.
> > >      c) CacheWAP (Javadoc documenting the web API)
> 
> I assume these two will be removed later:
> http://whiletaker.homeip.net/votorola/harvester/javadoc/votorola/s/wap/DiffC
> acheSS.html
> http://whiletaker.homeip.net/votorola/harvester/javadoc/votorola/s/wap/Harv
> estCacheSS.html 

Yes. Sooner than later. I can migrate the current Servlet first and adjust the 
feed, if you like me to.

> Leaving only this:
> http://whiletaker.homeip.net/votorola/harvester/javadoc/votorola/s/wap/Harve
> stWAP.html
> 
> (i) In the response, I would say "bites in order of parsed date,
> newest first" because it's not obvious.
> 
> (ii) I would link *all* of the fields to their javadocs, because they
> give useful information, and they're what the client is dealing with.

Done.
> 
> > > 3. Document the configuration of the Pipermail harvester.  The
> > > 
> > >    various harvesters should have similar forms of configuration,
> > >    but this cannot be required.  There are two major parts to the
> > >    
> > >    configuration:
> > >      a) User configuration in pollwiki, such as archive location
> 
> ok, if you like my latest: http://zelea.com/w/Special:RecentChanges
> 
> I added what you suggested: http://zelea.com/w/Concept:Forum_design
> 

Good, this should cut it for now. We will add/change it, if it is necessary 
for new Harvesters, I guess.

> 
> (i) This isn't really part of the theatre any more is it?  It's mainly
> a standalone service.  Maybe just add "voharvest" to the manual?
> http://zelea.com/project/votorola/s/manual.xht#Line
> 
> All that needs to be documented is the "voharvest" command itself.
> There's already a "vowebharvest" (though just a link), which I guess
> will be removed or renamed.
> 
> > > 5. Code it.
> 
> It seems ready to code - after you review my points above!
Sure. Anything missing still? I have updated the javadocs (1), manual.xht (2)
and theatre.xht (3). If this is it, I will commit it and link to this thread 
in the commit message.

conseo

(1) 
http://whiletaker.homeip.net/votorola/harvester/javadoc/index.html?votorola/a/diff/harvest/package-
summary.html
(2) http://whiletaker.homeip.net/votorola/harvester/manual.xht
(3) http://whiletaker.homeip.net/votorola/harvester/theatre.xht