Harvester Design

[conseo]

Hi,

> 
> We already have forums.  http://zelea.com/w/Concept:Forum
> So the candidate's position page will have one or more Forum
> properties that point to all the forums in which discussions are
> happening.  http://zelea.com/w/Property:Forum
> 
>   Position
>     -> Forum
> 
> Forums have a property defining the archive location:
> http://zelea.com/w/Property:Archive_URL
> All we need to add is an archive format:
> 
>   Forum
>     -> Archive URL
>     -> Archive format
> 

Good. I think we need a Forum type (here "Mailman") as well. In fact we don't 
always need to know the Archive, because if we configure the base-url of 
Mailman, then we can access the Archive from there. Yet configuring both would 
be sanest. Detectors only know Forums, while Harvesters need to understand 
different kind of archives for one Forum type. Or do you want to hardwire it 
around archive type ("Pipermail") for now?
 
>   voharvest clear FORUM   - clear FORUM from the cache
>   voharvest detect        - run the harvest detectors
>   voharvest harvest FORUM - harvest any new messages

Added to docs. (1)

> 
> Whatever the diff feed needs to run, because that's currently your
> only client.  Later you can extend the API to support the talk track
> if it needs additional request methods.
> 
> So 2 is almost done.  3a is pretty much done if you agree.  That
> 
> leaves 3b and 4 to document:
> > > 3. Document the configuration of the Pipermail harvester.  The
> > > 
> > >     various harvesters should have similar forms of configuration,
> > >     but this cannot be required.  There are two major parts to the
> > >     
> > >     configuration:
> > >       a) User configuration in pollwiki, such as archive location
> > >       b) Administrative configuration on server

3 b) For Pipermail (and Irssi) we don't need server side configs, we will 
fetch all configs out of the Wiki (as already discussed on IRC). 

4) see above. 

Javadocs should reflect all proposed changes now. (2)

conseo

(1) http://whiletaker.homeip.net/votorola/harvester/theatre.xht

(2) 
http://whiletaker.homeip.net/votorola/harvester/javadoc/votorola/a/diff/harvest/package-
summary.html