Announcement Announcement Module
No announcement yet.
design question: passing context Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • design question: passing context

    I have a question about passing context in message-oriented systems (isn`t related to JMS in particular).

    I have to following components:

    Fetcher (responsible for downloading pages)
    OutlinkExtractor (responsible for extracting links from pages)
    LinkDbWriter (responsible for writing the links to the db).

    The components are connected with channels and messages can be passed between the components.

    More detail:
    The Fetcher retrieves Download objects from the db. In the download object the url can be found, and also the depth (how many links where following from the starting point).

    The OutlintExtractor retrieves FetchResults objects (where the page content (string) is stored)). After is receives a FetchResults, it extracts all the outlinks and send them to the LinkDbWriter (it sends it to a channel that is connected to the LinkDbWriter)

    The LinkDbWriter creates a Download object with the link to download and the link depth (one more than the page where the link was on).

    The problem:
    I need to pass information from the fetcher to the writer, without imposing any restrictions on the OutlinkExtractor. The information I need to pass is the depth. I need to depth so can control how deep something is crawled. The new depth will be the depth of the page where the link is found + 1.

    How can I pass information from A (through B) to C without imposing any restrictions on B? The OutlinkExtractor has to be independant of the other components so I can reuse it in a totally different context.

    My Solutions so far:

    Some kind of map that is passed between the messages. The Fetcher stores the depth in the map, passes it to the OutlinkExtractor. The OutlinkExtractor retrieves the link, drops the link (with the map) in the following channel and the LinkDbWriter retrieves the depth and everybody is happy.

    But well.. if feels not that great. I have no control if the data is passed correctly and it feels realy 'loose'.

    The messages that B needs should be abstract. So the Fetcher can create a subclass with the depth field and the linkdbwriter can cast the object to the concrete implementation an retrieve the depth field. This solution feels more solid (just thought of it).

    So.. how do you pass context between messages?

    the second solution doesn`t hold. The Extractor creates new objects that don`t know about the depth field (I could extract the object creation with some kind of factory.. but the complexity increases).