Announcement Announcement Module
Collapse
No announcement yet.
Auditing systems based on integration Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Auditing systems based on integration

    I am beginning a project that is going to span hundreds of machines. I want to integrate something similar to log back audit into the system to handle audit entries and such into a central logging system that writes to a Hadoop database. I like the idea of using a messaging platform for this, so far I have been very impressed with mqtt for its raw speed.

    Has anyone tried doing an auditing system where I can haphazardly create events like "attempted login, invalid password", "login succeed", "opened dashboard", "method x took y ms to run" etc with integration?

    Is integration the best platform to try this with?

    I would be interested into building this right into the integration framework if the interest exists and there is an agreement it should be there.

  • #2
    Folks have done similar things with RabbitMQ (and even JMS) before.

    We'd certainly be interested in an MQTT implementation for Spring Integration.

    For information see https://github.com/SpringSource/spri...ONTRIBUTING.md

    We have an extensions repository for things like that...

    https://github.com/SpringSource/spri...ion-extensions

    ...and would be happy to accept a contribution.

    Finally, I thought I should point out there is an MQTT plugin for RabbitMQ... http://www.rabbitmq.com/blog/2012/09/12/mqtt-adapter/

    Comment


    • #3
      ok, great, i will start looking into the MQTT thing. Im not sure the raw speed can be replicated on a java platform but it will be fun to try.

      My big project would be building a unified auditing system. Thats my major problem. Something that can be used by just about anything, apps, arduino's, iphone, androids, etc. A standard auditing framework that could be distributed as an inter compatible method of auditing applications. Something that has a standard dashboard system to search and view the audits, maybe write some python scripts that can be uploaded and executed by the dashboard to analyze the data. Thats my big goal. Im still trying to figure out a way to do it. I really don't want to start the rest of the application without this piece. Thats why I was hoping I could use Integration as a starting point, then maybe get a few of you Spring masters interested and involved and come out with something really nice. I have some bar napkin designs done, thats about it. for the time being I will try my hand at the MQTT and see if I can get anything interesting to happen there.

      Comment


      • #4
        FYI, we now have a spring-integration-mqtt project in the Spring Integration Extensions repo. It uses the Eclipse Paho client; we'll be releasing a milestone shortly, as soon as the Paho mavenization details are finalized.

        Comment


        • #5
          Oh thats awesome. My project stalled a bit but its certain im going to use integration as a baseline for the auditing system i want to implement. the mqtt interface will be beautiful. Hopefully the code is clean enough to get the raw speed that mqtt was designed for. But if its slow i can jump in and take a look, most times a second set of fresh eyes helps a lot.

          And im going to build my audting system as an extension, who knows, you may want to include it.

          Comment


          • #6
            Hi,

            I am trying to have a look at the spring-integration-mqtt project but I am struggling to get going with it.

            So far, I have downloaded the spring-integration-mqtt-1.0.0.M1.jar and org.eclipse.paho.client.mqttv3.jar files.

            I have then created a simple spring-integration.xml file where I have declared:

            Code:
            xmlns:int-mqtt="http://www.springframework.org/schema/integration/mqtt"
            and

            Code:
            xsi:schemaLocation="http://www.springframework.org/schema/integration/mqtt http://www.springframework.org/schema/integration/mqtt/spring-integration-mqtt.xsd"
            However, when I try to include an outbound adapter as:

            Code:
            <int-mqtt:outbound-channel-adapter id="mqttOutboundAdapter" ... />
            the adapter is not recognized.

            would there happen to be an example anywhere online which I can peek at to see what I am still missing in my config and/or dependencies?

            Thanks in advance for any help on this.
            PM.

            Comment


            • #7
              Hi!
              the adapter is not recognized.
              Isn't it recognized just only by your IDE or do you get some error when you run your app?
              Show StackTrace, please.

              Take care,
              Artem

              Comment


              • #8
                Hi Cleric,

                It is not recognized by the IDE. The error is:

                cvc-complex-type.2.4.c: The matching wildcard is strict, but no declaration can be found for element 'int-mqtt:outbound-channel-adapter'.

                Something like <int:channel id="ch1"/> is recognized as expected however.

                Regards,
                PM.

                Comment


                • #9
                  cvc-complex-type.2.4.c: The matching wildcard is strict,
                  Try to find solutions in this forum or via Google.
                  And take a look here: http://forum.springsource.org/showth...ngs-And-Errors

                  Comment


                  • #10
                    Thanks for the helper link. I've added Spring Nature to my project and now it is showing me the following error:

                    Build path is incomplete. Cannot find class file for org/eclipse/paho/client/mqttv3/IMqttDeliveryToken.

                    I've checked and, the IMqttDeliveryToken.class is missing from that package in my paho client jar (org.eclipse.paho.client.mqttv3.jar). There is an MqttDeliveryToken.class implementation class however. Should I be using another version of the paho client library instead?

                    Cheers,
                    PM.

                    Comment


                    • #11
                      Cannot find class file for org/eclipse/paho/client/mqttv3/IMqttDeliveryToken.
                      H-m. I'm not well with mqtt, but I see Gary compiled it with this: https://github.com/SpringSource/spri...ild.gradle#L63
                      HTH

                      Comment


                      • #12
                        Yep, "org.eclipse.paho:mqtt-client:0.2.1" in that build file resolves the missing dependencies.

                        Thanks again for your help!

                        Comment

                        Working...
                        X