Announcement Announcement Module
Collapse
No announcement yet.
no openid attrs from Yahoo Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • no openid attrs from Yahoo

    The sample openid app (from Git master tip) does not retrieve any attributes from AOL. It doesn't retrieve any attributes from AOL or Wordpress either, but the app is configured out-of-the-box to retrieve attributes from AOL.

    The sample works perfectly for me with Google, OpenID and myOpenID providers.

    Web searches indicate that Yahoo uses the axschema for attributes, and that is what is configured for the sample app, but only nulls values are retrieved.

    Authentication is granted. I verified on the Yahoo site that Yahoo is configured to give my personal details to Relying parties.

    Web searches show that there are peculiarities for requirements for "count" parameters, and my own tests show peculiarities for "required" parameters, but whereas Google and OpenID do return the attributes with at least one combination of these settings, Yahoo works for no combination of them.

    (I have also tried to fetch the first/last/full/email attrs from AOL and Wordpress by asking for them with both axs and openid schema names, and have had no success with them either. Since my app requires third-party-validated email addresses, I guess the only, distasteful, workaround is to disable all providers other than Google + OpenId (+ MyOpenID, which is just a specific OpenID domain if I understand that correctly).
    Last edited by blaine; Mar 25th, 2011, 01:29 PM. Reason: corrected source code source

  • #2
    Originally posted by blaine View Post
    The sample openid app (from Git master tip) does not retrieve any attributes from AOL. It doesn't retrieve any attributes from AOL or Wordpress either, but the app is configured out-of-the-box to retrieve attributes from AOL.
    What attributes are you expecting to get from AOL and Wordpress? Can you provide any documentation that they return these attributes?

    Originally posted by blaine View Post
    Web searches indicate that Yahoo uses the axschema for attributes, and that is what is configured for the sample app, but only nulls values are retrieved.
    I am able to get email and and full name returned from Yahoo.
    Originally posted by blaine View Post
    Web searches show that there are peculiarities for requirements for "count" parameters, and my own tests show peculiarities for "required" parameters, but whereas Google and OpenID do return the attributes with at least one combination of these settings, Yahoo works for no combination of them.
    The OP is not suppose to fail if the attribute is not available and leaves it up to the RP to determine how to handle the situation if the attribute is not returned. The sample application is very lenient, but you could customize the AuthenticationProvider to validate that you get the attributes you require back.

    Originally posted by blaine View Post
    (I have also tried to fetch the first/last/full/email attrs from AOL and Wordpress by asking for them with both axs and openid schema names, and have had no success with them either. Since my app requires third-party-validated email addresses, I guess the only, distasteful, workaround is to disable all providers other than Google + OpenId (+ MyOpenID, which is just a specific OpenID domain if I understand that correctly).
    I'm not sure exactly what you are trying to accomplish with requiring the OpenID provider to return an email address. Can you elaborate on this? Couldn't you just have the user type in an email if the OP did not provide one?

    Cheers,

    Comment


    • #3
      Originally posted by rwinch View Post
      What attributes are you expecting to get from AOL and Wordpress? Can you provide any documentation that they return these attributes?
      No. I provided that as further information in case somebody else knows that they do. I am concentrating on Yahoo because that is the one that the example expects to supply attributes.

      Originally posted by rwinch View Post
      I am able to get email and and full name returned from Yahoo.
      That is very good to know.

      I am thinking that this is due to not RP (or RP realm) yadis-validated. I found discussions and HOWTOs from a couple years ago saying that Yahoo will warn the user if the RP is not validated and I was presented with no warning at all, so I figured it was fully satisfied, but perhaps not.

      I set off to set up Yadis validation but ran into difficulties because my public site's home page does a redirect, and has a production biz app on it anyways, so I need to specify a non-default realm URL and I am working on problems with that (discussion about that in a separate thread here).

      Originally posted by rwinch View Post
      The OP is not suppose to fail if the attribute is not available and leaves it up to the RP to determine how to handle the situation if the attribute is not returned. The sample application is very lenient, but you could customize the AuthenticationProvider to validate that you get the attributes you require back.
      Sorry about the ambiguity there. What I meant to say was that Yahoo returns me no attribute values for all combinations of "count" +"required" + the 2 common attribute name spaces. As I said above, it does grant me access. It fails to return any attributes. (Specifically, the attribute list in the Spring object has all null values).


      Originally posted by rwinch View Post
      I'm not sure exactly what you are trying to accomplish with requiring the OpenID provider to return an email address. Can you elaborate on this? Couldn't you just have the user type in an email if the OP did not provide one?
      1. Not bother the user with unnecessary work, especially when the user may know that the app should be able to get the information itself. Analogous to having the user enter the current time: some users will think, "Why isn't this program smart enough to get the time itself"
      2. Generality because my app also supports LDAP and JDBC authentication providers and I require these to manage and supply email addresses
      3. Simplicity because I want to trust the provider to have verified email addresses and not have my app assume the responsibility to code round-trip confirmation tests to prevent mistakes or exploits.


      Originally posted by rwinch View Post
      Cheers,

      Comment


      • #4
        Originally posted by blaine View Post
        I am thinking that this is due to not RP (or RP realm) yadis-validated. I found discussions and HOWTOs from a couple years ago saying that Yahoo will warn the user if the RP is not validated and I was presented with no warning at all, so I figured it was fully satisfied, but perhaps not.
        ...
        Sorry about the ambiguity there. What I meant to say was that Yahoo returns me no attribute values for all combinations of "count" +"required" + the 2 common attribute name spaces. As I said above, it does grant me access. It fails to return any attributes. (Specifically, the attribute list in the Spring object has all null values).
        So it sounds like you believe the problem to be related to the custom realm? I have responded on the other thread, so if this is the case just acknowledge that I am right here and keep that discussion on the other thread. If I am wrong please clarify for me.

        Originally posted by blaine View Post
        [*]Not bother the user with unnecessary work, especially when the user may know that the app should be able to get the information itself. Analogous to having the user enter the current time: some users will think, "Why isn't this program smart enough to get the time itself"
        Fair enough, but other users might think why can't I login with AOL?

        Originally posted by blaine View Post
        [*]Generality because my app also supports LDAP and JDBC authentication providers and I require these to manage and supply email addresses
        Couldn't you verify the email on your own?

        Originally posted by blaine View Post
        [*]Simplicity because I want to trust the provider to have verified email addresses and not have my app assume the responsibility to code round-trip confirmation tests to prevent mistakes or exploits.
        Fair enough but keep in mind that anyone can write an OpenID provider which can return any email address. So if I wanted to trick your application into thinking that I owned a different email address I could just stand up my own OP. Additionally, keep in mind that the attribute exchange protocol does not require signing of the attributes exchanged. As I recall the openid4java library (which is the impl for openid) takes this very literally and by default does not require that the email attribute is signed. This means that I don't even need to stand up an OpenID Provider...I can just use something like tamper data to specify a different email address. In short, you cannot really trust that the email address has been validated. To fix this issue you would have to customize openid4java (see SEC-1711)

        So, in my opinion, you might as well try to get the email address and if you can't allow them to specify it. Either way you should likely validate the email address yourself.

        Cheers,
        Last edited by Rob Winch; Apr 8th, 2011, 12:13 AM.

        Comment


        • #5
          Originally posted by rwinch View Post
          So it sounds like you believe the problem to be related to the custom realm? I have responded on the other thread, so if this is the case just acknowledge that I am right here and keep that discussion on the other thread. If I am wrong please clarify for me.
          I don't know, I just think it is related to custom realm by process of elimination because you can fetch attrs and I can't. I will change my belief if I get yadis validation working and I still can't fetch attrs.

          But yes, that thread is for that discussion.

          Originally posted by rwinch View Post
          Fair enough, but other users might think why can't I login with AOL?
          Hopefully AOL and the other providers will return the attrs once I get yadis validation working, and this will then not be an issue.

          Originally posted by rwinch View Post
          Couldn't you verify the email on your own?
          I could. I could also check the local weather. (I'm not trying to be sarcastic, just trying to concisely answer by analogy). I want my app to do as little as possible to achieve my design goals; I don't want unnecessary application complexity; and I don't want to bother the user with confirmation emails and such if there isn't a need to.

          If I can't satisfy the design goal of basic good security, then I'll have to add complexity. Since I am already using the openid service, I would prefer to use a feature that is already there. The user will already have validated the addr. This is precisely why both openid attribute schemas have an attribute for email address and I would like to use the feature.

          Originally posted by rwinch View Post
          Fair enough but keep in mind that anyone can write an OpenID provider which can return any email address. So if I wanted to trick your application into thinking that I owned a different email address I could just stand up my own OP. Additionally, keep in mind that the attribute exchange protocol does not require signing of the attributes exchanged. As I recall the openid4java library (which is the impl for openid) takes this very literally and by default does not require that the email attribute is signed. This means that I don't even need to stand up an OpenID Provider...I can just use something like tamper data to specify a different email address. In short, you cannot really trust that the email address has been validated. To fix this issue you would have to customize openid4java.

          So, in my opinion, you might as well try to get the email address and if you can't allow them to specify it. Either way you should likely validate the email address yourself.
          I am going to restrict OPs to those specified in the app. We are discussing that here: http://forum.springsource.org/showthread.php?t=106205

          Regarding signing of the email attr, that's news to me. I thought that all of that would be signed like with OAuth. Maybe I will need to validate the addrs. Even if I can't depend on the security of the attributes, I want to use the openid attribute feature, not just the email addr attribute, at least to provide default or starting values.

          Originally posted by rwinch View Post
          Cheers,
          Cheers.
          Last edited by blaine; Mar 30th, 2011, 09:09 AM.

          Comment


          • #6
            Originally posted by blaine View Post
            I could. I could also check the local weather. (I'm not trying to be sarcastic, just trying to concisely answer by analogy). I want my app to do as little as possible to achieve my design goals; I don't want unnecessary application complexity; and I don't want to bother the user with confirmation emails and such if there isn't a need to.

            If I can't satisfy the design goal of basic good security, then I'll have to add complexity. Since I am already using the openid service, I would prefer to use a feature that is already there. The user will already have validated the addr, and this is precisely why the openid feature is provided.
            I'm not trying to be restrictive in what your application wants. My goal was to attempt to be informative. It sounded as though you were wanting to ensure you had a valid email. I just wanted to make sure certain things had been taken into consideration in hopes that your solution is secure. If you do not want this complexity you have a few steps (some of which it sounds like you have already considered). You will need to whitelist the OPs that you believe to be validating the email address (sounds like you have done this). You will want to ensure that the OP not only provides the email address, but also signs it. You will also need to ensure that when you receive the response that you require the email to be signed. The disadvantage I see with this is if a user wants to use an OP that isn't whitelisted they are inconvenienced with having to sign up for a different OP. This may not be an issue for your application (it really depends on its purpose, requirements, etc). Again I was just attempting to ensure you had all the information needed to make your application work as well as possible.

            Originally posted by blaine View Post
            Regarding signing of the email attr, that's news to me. I thought that all of that would be signed like with OAuth. Maybe I will need to validate the addrs.
            OAuth and OpenID are quite different in many respects. The OpenID 2.0 Specification states:

            openid.signed

            Value: Comma-separated list of signed fields.

            Note: This entry consists of the fields without the "openid." prefix that the signature covers. This list MUST contain at least "op_endpoint", "return_to" "response_nonce" and "assoc_handle", and if present in the response, "claimed_id" and "identity". Additional keys MAY be signed as part of the message. See Generating Signatures.

            For example, "op_endpoint,identity,claimed_id,return_to,assoc_h andle,response_nonce".
            The attribute exchange specification does not state any addition fields are required to be signed, so it is up to the implementor to determine this. You can find that openid4 java code (which is what back Spring Security's OpenID implementation) decided to be lenient by default.

            Cheers,

            Comment


            • #7
              I have implemented RP XRDS realm validation according to http://blog.nerdbank.net/2008/06/why...enid-site.html .

              Please try my proof-of-concept app by using the Administer Site link at http://admc.com/webapp-base . After logging in, you can check whether your email or name attributes were retrieved by going back to home page and clicking link "Spring Security-aware". You can see that for Google, myOpenID, and OpenID providers it reports your full name and email address-- and this worked fine before I set up XRDS validation.

              For Yahoo, the login process appears to work perfectly but my debug log shows that every requested attribute mapping has value null (and my "Spring Security-aware" page displays no full name nor email addr).

              WordPress behavior is the same, but I don't know which attributes WordPress should support.

              I did more searching about AOL. It should support the axschema attributes, as described at http://practicalid.blogspot.com/2010...-live-aol.html . I was hopeful for AOL because the login page at AOL even displays my email address and says that it will share. Unfortunately, it yields all nulls as attribute values, including for the email address that it said it would share. Same symptoms as for Yahoo and WordPress.

              (After I set up XRDS, the AOL page was complaining that it failed to validate my app, but it would still grant access to my site. With my latest test, however, it isn't displaying the warning. I don't know if it is just slow to pick up changes or if it still considers my site non-validated but considers this one user to be already warned).

              I am quite confused as to why Rob's usage of Spring Security + openid retrieves attributes from Yahoo, but mine does not. I don't see how my openid wiring or my attribute retrieval code could be wrong because I am able to successfully retrieve both axschema and openid schema attributes from Google and MyOpenId respectively. My AuthenticationUserDetailsService dumps the attributes to a log file before I touch them, so the problem can not be caused by me extracting them incorrectly from the OpenIDAuthenticationToken.getAttributes(). Since every provider works just fine to grant access to my site, the links between my login page and the provider sites must be right.

              Rob, with respect to your web app tha successfully fetches attributes from Yahoo, is your app running with SSL? Do you have XRDS validation set up for your openid realm?

              Let me know if my XML configs, log files, or anything else, would be useful. If you think that XRDS may be at fault, the realm home is at http://admc.com/webapp-base/openidRealm/ .
              Last edited by blaine; Apr 3rd, 2011, 03:04 PM. Reason: offer technical details

              Comment


              • #8
                Originally posted by rwinch View Post
                ...
                OAuth and OpenID are quite different in many respects...
                Thanks for the great info.

                I'm now going to attempt to implement a registration process that validates email address and site user name.

                Comment


                • #9
                  Originally posted by blaine View Post
                  I am quite confused as to why Rob's usage of Spring Security + openid retrieves attributes from Yahoo, but mine does not.
                  ...
                  Rob, with respect to your web app tha successfully fetches attributes from Yahoo, is your app running with SSL? Do you have XRDS validation set up for your openid realm?
                  I ran the Spring Security Sample OpenID application from master using gradle jettyRun from the openid sample directory. I was running using localhost, over http, without XRDS validation. Does it work for you using the sample application?

                  If you are able to provide a minimal example and instructions so I can reproduce the problem, I don't mind taking a look at it to see if I can figure out what is happening.

                  Comment


                  • #10
                    After setting my Yahoo user name in other places (you can set it in at least 3 places), the SS openid sample app does fetch the namePerson attribute. That's the only good thing that I have to report.

                    My app, which requests the same exact attributes gets nothing back for namePerson. Thinking of differences between my app and the sample app, I am going to see if the difference is caused by different versions of openid4java and related libraries, and newer openid-selector components.

                    With respect to the email address, it seems that Yahoo is not trying to send my email address, because whereas the Yahoo login page says that it will share my user name, it says nothing about email address. I see in some Yahoo documentation that it says it will share the user's Yahoo email address. Doesn't make sense to me to withhold validated email addresses because they happen to be non-Yahoo, but... When you run the sample app and it reports your email address from Yahoo, it is a yahoo domain email address?

                    FYI, I get the same results regardless of whether I run the sample app through gradle Jetty or if I build the war with gradle and deploy to Tomcat. Also same if I deploy it locally or to my public server, and whether I access it with localhost URL or a specific host name URL.

                    I'm out of time for today before accomplishing much again. Day job's becoming a night job too and not leaving me much time for this.

                    Comment


                    • #11
                      Originally posted by blaine View Post
                      After setting my Yahoo user name in other places (you can set it in at least 3 places)
                      Do you mind elaborating what you mean by this? For example, where did you have to set the name for it to work? Was it a Yahoo setup problem, Spring Security setup/code problem, etc? This information might be useful to others that are having the same problem.

                      Originally posted by blaine View Post
                      When you run the sample app and it reports your email address from Yahoo, it is a yahoo domain email address?
                      Yes. The email returned by yahoo is an @yahoo.com email address.

                      Comment


                      • #12
                        Originally posted by rwinch View Post
                        Do you mind elaborating what you mean by this? For example, where did you have to set the name for it to work? Was it a Yahoo setup problem, Spring Security setup/code problem, etc? This information might be useful to others that are having the same problem.
                        Yahoo setup. Just like at Google, there are different Yahoo account types underneath, and though they are pretty successful at giving the facade that a Yahoo account is a Yahoo account, the subsystems aren't all that well integrated when you need to control the details. There are entirely different account setup screens that you get to from navigating from different places.

                        Originally posted by rwinch View Post
                        Yes. The email returned by yahoo is an @yahoo.com email address.
                        Ok. Hopefully within a few hours I'll create my own Yahoo email account and try things out with that.

                        UPDATE: Sample app works when there is a *@yahoo.com email addresses associated with the Yahoo account!
                        Much simpler troubleshooting task now. I need to see how my app differs from the sample app.
                        Last edited by blaine; Apr 6th, 2011, 09:24 PM. Reason: added new finding

                        Comment


                        • #13
                          I updated a previous post on this thread to include a link to SEC-1711. This ticket enhances Spring Security to guarantee attribute exchange parameters are signed. This repost is so that those subscribed to the thread get notified (I don't think notifications are done for edits).

                          Cheers,

                          Comment


                          • #14
                            Solved

                            It was very time consuming to test this because Yahoo's provider service is extremely slow lately and has been timing out today about 5 out of 6 attempts. This had nothing to do with me because I verified that it is even occurring when logging in to sourceforge.net with a Yahoo account.

                            My problem was that I made an unsupported assumption that OpenIDAttribute names are client-side and arbitrary. I came to this incorrect conclusion base on
                            1. With Google and MyOpenId providers, attr. names apparently are arbitrary, because these work great when I use my own made-up names.
                            2. The several lists of supported openid attributes that I have been using all list just the URIs like "http://axschema.org/contact/email" which correspond to OpenIDAttribute.type properties.
                            3. There is no obvious correspondence between "type" values and "name" values in the Spring Security openid sample app. The names struck me as informal names by convention or by the whim of the original author of the sample app.

                            When I use the names that are used in the sample app, Yahoo sends back the correct values for the paired type.

                            I also got AOL working. AOL also requires specific "names". Email name and addr same as Yahoo. Full name is not available, but you can pull the namePerson/friendly value by using attr name "nickname".
                            Last edited by blaine; Apr 10th, 2011, 09:11 AM. Reason: report detail of google/MyOpenId behavior

                            Comment

                            Working...
                            X