From elly at cs.cornell.edu Mon Aug 6 16:11:03 2007 From: elly at cs.cornell.edu (Elly Cramer) Date: Mon Aug 6 16:25:54 2007 Subject: [Ndr-users] NCS and application information Message-ID: <04745CBF7A8C024C9267D85521D18BEB01A7503B@EXCHANGE1.cs.cornell.edu> Katy, I met with Tim, DeanX, and Aaron about the info needed for collections using the NDR API applications vs OAI to provide metadata. We concluded that it would be very useful to have the collection provide their "Application Public Key". This could be a textarea input field in your form. It is considered more secure for the collection provider to give us the public key (in your collection form) than for us to create and email it to them (not good practice). The application public key lives in the collection's agent. I'm sending this to the ndr-users list (which I added you to) since I wanted to share this information as it relates to the NCS and creating agents for new collections. I believe Aaron and Jonathan had some discussion about this last week. I realize this topic will need further discussion, but thought I would get the conversation started here. - elly From ostwald at ucar.edu Tue Aug 7 11:54:43 2007 From: ostwald at ucar.edu (Jonathan Ostwald) Date: Tue Aug 7 11:54:48 2007 Subject: [Ndr-users] NCS and application information In-Reply-To: <04745CBF7A8C024C9267D85521D18BEB01A7503B@EXCHANGE1.cs.cornell.edu> References: <04745CBF7A8C024C9267D85521D18BEB01A7503B@EXCHANGE1.cs.cornell.edu> Message-ID: <46B895C3.9070801@ucar.edu> Elly (and all) - I believe there was an additional conversation in Boulder last week (that didn't include me or Katy) in which it was decided that each collection managed in the NCS should have it's own Agent. Earlier in the week, in the meeting involving Aaron, Mike W., Katy and me, we touched on this topic but didn't get into any depth. So before talking about managing the public keys of the collections, I would like to understand how the big picture of how the "Collection Agent" should be represented in the NDR and what the respective roles are of the NCS agent and Collection Agent. I'm not sure how collections should be represented in the NDR using a per-Collection Agent, but I am guessing that the following is the way to go: - the Collection Agent would "own" both the MDP and the Aggregator (via "metadataProviderFor", and "aggregatorFor" relationships), - and the NCS would be a "trusted agent" via "authorizedToChange" relationships in the MDP and Aggregator, which it would create. ----- Assuming we've figured out how to represent a collection in the NCS that is associated with a Collection Agent, but "changable" by the NCS, let's now talk about the process behind having public keys supplied by the collections. Is the following along the lines of what you envision? 1 - the collection management folks (who use the NCS to manage collection records) would generate the public / private key pairs. (i assume that the collection owners may not be willing/able to to do this). 2 - the public key is entered in the a field of the ncs_collect record managed in NCS and then written to NDR as a datastream within a Collection Agent (which the NCS would create directly as opposed to the NSDL on behalf of NCS?). 3 - the private key is stored within the application (need to work out how this would be done, but it could be stored as part of the Collection Configuration). Questions: 1 - When does the private key even come into play, since the NCS Agent is the authorized Agent that is making changes in the NDR? And if the CollectionAgent CAN make changes to Objects in the NDR, it seems we have lost the concurrency (and some integrity) protection the NCS provides when it is the sole means for modifying the Metadata Objects in the NDR ... 2 - Can the NCS, as a Trusted Application, create an Agent and populate it's important properties? Yesterday I was able to create an Agent object in the NDR, but i couldn't seem to set the DC stream. (I didn't try to set the Public Key) thanks, Jonathan Elly Cramer wrote: > Katy, > > I met with Tim, DeanX, and Aaron about the info needed for collections using > the NDR API applications vs OAI to provide metadata. We concluded that it > would be very useful to have the collection provide their "Application Public > Key". This could be a textarea input field in your form. > > It is considered more secure for the collection provider to give us the > public key (in your collection form) than for us to create and email it to > them (not good practice). > > The application public key lives in the collection's agent. I'm sending this > to the ndr-users list (which I added you to) since I wanted to share this > information as it relates to the NCS and creating agents for new collections. > I believe Aaron and Jonathan had some discussion about this last week. > > I realize this topic will need further discussion, but thought I would get > the conversation started here. > > - elly > > > _______________________________________________ > ndr-users mailing list > ndr-users@comm.nsdl.org > http://comm.nsdl.org/mailman/listinfo/ndr-users -- Jonathan Ostwald Digital Learning Sciences University Corporation for Atmospheric Research phone: 303-497-2661 e-mail: ostwald@ucar.edu http://www.dlese.org From birkland at cs.cornell.edu Tue Aug 7 13:19:18 2007 From: birkland at cs.cornell.edu (Aaron Birkland) Date: Tue Aug 7 13:19:04 2007 Subject: [Ndr-users] NCS and application information In-Reply-To: <46B895C3.9070801@ucar.edu> References: <04745CBF7A8C024C9267D85521D18BEB01A7503B@EXCHANGE1.cs.cornell.edu> <46B895C3.9070801@ucar.edu> Message-ID: <46B8A996.2060302@cs.cornell.edu> Jonathan, > I'm not sure how collections should be represented in the NDR using a > per-Collection Agent, but I am guessing that the following is the way > to go: > > - the Collection Agent would "own" both the MDP and the Aggregator > (via "metadataProviderFor", and "aggregatorFor" relationships), > > - and the NCS would be a "trusted agent" via "authorizedToChange" > relationships in the MDP and Aggregator, which it would create. > Yes, that is correct > 1 - When does the private key even come into play, since the NCS Agent > is the authorized Agent that is making changes in the NDR? And if the > CollectionAgent CAN make changes to Objects in the NDR, it seems we > have lost the concurrency (and some integrity) protection the NCS > provides when it is the sole means for modifying the Metadata Objects > in the NDR ... The keys are only useful for users/applications that wish to initiate actions in the NDR themselves. If the NCS is managing various collections (with their respective agents, and all 'authorizedToChange' relationships pointing to the NCS agent), then each individual agent does *not* have to have a public key in the NDR. The NCS is the only application that is doing anything in the NDR, so it is the only one that needs a key pair. Now suppose that one of the collection agents *did* have a key pair. That means that this agent can "log in" and make changes to the NDR. However, if this agent has any underlying Aggregators or MetadataProviders that have 'authorizedToChange' pointing exclusively to the NCS agent, that agent cannot modify the contents of those objects unless the NCS agent wills it (i.e. modifies the objects to have authorizedToChange point to their own Agent). So, it would be reasonable *not* to give these collection agents a key pair unless they want some other non-NCS-related access to the NDR, or if you were creating an agent that would represent the controlling agent of another deployment of the NCS. > 2 - Can the NCS, as a Trusted Application, create an Agent and > populate it's important properties? Yesterday I was able to create an > Agent object in the NDR, but i couldn't seem to set the DC stream. (I > didn't try to set the Public Key) Yes, you should be able to, though there are some constraints on the content of the DC field that are imposed by Fedora (since it is actually a Fedora "special" datastream). The biggest: it must contain only unqualified dublin core. -Aaron From birkland at cs.cornell.edu Tue Aug 7 13:35:29 2007 From: birkland at cs.cornell.edu (Aaron Birkland) Date: Tue Aug 7 13:35:12 2007 Subject: [Ndr-users] NCS and application information In-Reply-To: <46B895C3.9070801@ucar.edu> References: <04745CBF7A8C024C9267D85521D18BEB01A7503B@EXCHANGE1.cs.cornell.edu> <46B895C3.9070801@ucar.edu> Message-ID: <46B8AD61.7090002@cs.cornell.edu> Come to think of it, I believe another (more plausible) use case for giving the NCS the ability to define a public key for the collection agent is if the NCS is used only to create (but not manage) a collection. For example, we 'manually' edited the ExpertVoices agent to include a public key, since it would use the API itself to manage its own contents. We could keep doing that, but it may be convenient if the NCS could do it. Sorry, I should have remembered that case. That doesn't change the fact that in 99% of cases, NCS would probably *not* want to put a key in the collection's agent, but the ability to do so is something we may want to think about at some point. -Aaron > > 1 - the collection management folks (who use the NCS to manage > collection records) would generate the public / private key pairs. (i > assume that the collection owners may not be willing/able to to do this). > From ostwald at ucar.edu Mon Aug 13 12:18:18 2007 From: ostwald at ucar.edu (Jonathan Ostwald) Date: Mon Aug 13 12:18:21 2007 Subject: [Ndr-users] escaped markup in NDR request Message-ID: <46C0844A.1080605@ucar.edu> NDR-test does not allow escaped markup (e.g., "<") in the inputXML parameter. Is this intentional? thanks, Jonathan ----------------------- < =============== proxyResponse inputXML parameter is not well-formed xml. -- Jonathan Ostwald Digital Learning Sciences University Corporation for Atmospheric Research phone: 303-497-2661 e-mail: ostwald@ucar.edu http://www.dlese.org From birkland at cs.cornell.edu Mon Aug 13 13:37:00 2007 From: birkland at cs.cornell.edu (Aaron Birkland) Date: Mon Aug 13 13:43:42 2007 Subject: [Ndr-users] escaped markup in NDR request In-Reply-To: <46C0844A.1080605@ucar.edu> References: <46C0844A.1080605@ucar.edu> Message-ID: <46C096BC.3020108@cs.cornell.edu> It should not matter (ie. it's not intentional - there should not be any limitation the value of properties). I haven't been able to reproduce the issue. What was api method you were using? I did a 'find' with this without error: < http://ithacasciencezone.com/ (it goes through the same wellformedness preprocessing step as the other calls). -Aaron Jonathan Ostwald wrote: > NDR-test does not allow escaped markup (e.g., "<") in the inputXML > parameter. > > Is this intentional? > > thanks, > > Jonathan > > ----------------------- > > xmlns="http://ns.nsdl.org/ndr/request_v1.00/" > xsi:schemaLocation="http://ns.nsdl.org/ndr/request_v1.00/ > http://ns.nsdl.org/schemas/ndr/request_v1.00.xsd" > schemaVersion="1.00.000"> > > > < > > > > > > =============== > proxyResponse > > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" > schemaVersion="1.00.000" > xsi:schemaLocation="http://ns.nsdl.org/ndr/response_v1.00/ > http://ns.nsdl.org/schemas/ndr/response_v1.00.xsd"> > > > inputXML parameter is not well-formed xml. > > > From ostwald at ucar.edu Mon Aug 13 14:12:48 2007 From: ostwald at ucar.edu (Jonathan Ostwald) Date: Mon Aug 13 14:12:52 2007 Subject: [Ndr-users] escaped markup in NDR request In-Reply-To: <46C096BC.3020108@cs.cornell.edu> References: <46C0844A.1080605@ucar.edu> <46C096BC.3020108@cs.cornell.edu> Message-ID: <46C09F20.9000802@ucar.edu> i was using a "modifyMetadata" request: ndrtest.nsdl.org/api/modifyMetadata/2200/test.20070810180538329T jonathan Aaron Birkland wrote: > It should not matter (ie. it's not intentional - there should not be any > limitation the value of properties). I haven't been able to reproduce > the issue. What was api method you were using? I did a 'find' with > this without error: > > > > > < > http://ithacasciencezone.com/ > > > > > (it goes through the same wellformedness preprocessing step as the other > calls). > -Aaron > > > Jonathan Ostwald wrote: >> NDR-test does not allow escaped markup (e.g., "<") in the inputXML >> parameter. >> >> Is this intentional? >> >> thanks, >> >> Jonathan >> >> ----------------------- >> >> > xmlns="http://ns.nsdl.org/ndr/request_v1.00/" >> xsi:schemaLocation="http://ns.nsdl.org/ndr/request_v1.00/ >> http://ns.nsdl.org/schemas/ndr/request_v1.00.xsd" >> schemaVersion="1.00.000"> >> >> >> < >> >> >> >> >> >> =============== >> proxyResponse >> >> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" >> schemaVersion="1.00.000" >> xsi:schemaLocation="http://ns.nsdl.org/ndr/response_v1.00/ >> http://ns.nsdl.org/schemas/ndr/response_v1.00.xsd"> >> >> >> inputXML parameter is not well-formed xml. >> >> >> -- Jonathan Ostwald Digital Learning Sciences University Corporation for Atmospheric Research phone: 303-497-2661 e-mail: ostwald@ucar.edu http://www.dlese.org From birkland at cs.cornell.edu Mon Aug 13 15:50:14 2007 From: birkland at cs.cornell.edu (Aaron Birkland) Date: Mon Aug 13 15:50:22 2007 Subject: [Ndr-users] escaped markup in NDR request In-Reply-To: <46C09F20.9000802@ucar.edu> References: <46C0844A.1080605@ucar.edu> <46C096BC.3020108@cs.cornell.edu> <46C09F20.9000802@ucar.edu> Message-ID: <46C0B5F6.80608@cs.cornell.edu> That's interesting: I just tried it and it worked (I used your inputXML unmodified). See: http://ndrtest.nsdl.org/api/get/2200/test.20070201144359083T (note: in the next update of ndrtest code, the content model will be strictly enforced. For this particular request, myProp property will not be allowed unless it is in some other namespace (since it is not part of the basic NDR model)) For your situation, I am wondering if the ndr is truly being sent what you think it is. For example, is the entity reference being decoded before the request is sent to the repository? I typically use some sort of packet sniffer to verify exactly what is being sent down the wire. If you don't have access to one, let me know and I can log all inputXML values that are not well formed XML and see what the NDR thinks it is receiving. -Aaron Jonathan Ostwald wrote: > i was using a "modifyMetadata" request: > > ndrtest.nsdl.org/api/modifyMetadata/2200/test.20070810180538329T > > jonathan > > Aaron Birkland wrote: >> It should not matter (ie. it's not intentional - there should not be >> any limitation the value of properties). I haven't been able to >> reproduce the issue. What was api method you were using? I did a >> 'find' with this without error: >> >> >> >> >> < >> http://ithacasciencezone.com/ >> >> >> >> >> (it goes through the same wellformedness preprocessing step as the >> other calls). >> -Aaron >> >> >> Jonathan Ostwald wrote: >>> NDR-test does not allow escaped markup (e.g., "<") in the >>> inputXML parameter. >>> >>> Is this intentional? >>> >>> thanks, >>> >>> Jonathan >>> >>> ----------------------- >>> >>> >> xmlns="http://ns.nsdl.org/ndr/request_v1.00/" >>> xsi:schemaLocation="http://ns.nsdl.org/ndr/request_v1.00/ >>> http://ns.nsdl.org/schemas/ndr/request_v1.00.xsd" >>> schemaVersion="1.00.000"> >>> >>> >>> < >>> >>> >>> >>> >>> >>> =============== >>> proxyResponse >>> >>> >> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" >>> schemaVersion="1.00.000" >>> xsi:schemaLocation="http://ns.nsdl.org/ndr/response_v1.00/ >>> http://ns.nsdl.org/schemas/ndr/response_v1.00.xsd"> >>> >>> >>> inputXML parameter is not well-formed >>> xml. >>> >>> >>> > From ostwald at ucar.edu Tue Aug 14 17:28:25 2007 From: ostwald at ucar.edu (Jonathan Ostwald) Date: Tue Aug 14 17:28:28 2007 Subject: [Ndr-users] escaped markup in NDR request In-Reply-To: <46C0B5F6.80608@cs.cornell.edu> References: <46C0844A.1080605@ucar.edu> <46C096BC.3020108@cs.cornell.edu> <46C09F20.9000802@ucar.edu> <46C0B5F6.80608@cs.cornell.edu> Message-ID: <46C21E79.8010707@ucar.edu> Aaron - I've set up a sniffer on my machine here, but as far as i can tell, the stuff going out is as it should be ... So it would be very helpful if you could log the non-well-formed inputXML values. thanks! jonathan Aaron Birkland wrote: > That's interesting: I just tried it and it worked (I used your inputXML > unmodified). See: > http://ndrtest.nsdl.org/api/get/2200/test.20070201144359083T > > (note: in the next update of ndrtest code, the content model will be > strictly enforced. For this particular request, myProp property will > not be allowed unless it is in some other namespace (since it is not > part of the basic NDR model)) > > For your situation, I am wondering if the ndr is truly being sent what > you think it is. For example, is the entity reference being decoded > before the request is sent to the repository? I typically use some > sort of packet sniffer to verify exactly what is being sent down the > wire. If you don't have access to one, let me know and I can log all > inputXML values that are not well formed XML and see what the NDR thinks > it is receiving. > > -Aaron > > Jonathan Ostwald wrote: >> i was using a "modifyMetadata" request: >> >> ndrtest.nsdl.org/api/modifyMetadata/2200/test.20070810180538329T >> >> jonathan >> >> Aaron Birkland wrote: >>> It should not matter (ie. it's not intentional - there should not be >>> any limitation the value of properties). I haven't been able to >>> reproduce the issue. What was api method you were using? I did a >>> 'find' with this without error: >>> >>> >>> >>> >>> < >>> http://ithacasciencezone.com/ >>> >>> >>> >>> >>> (it goes through the same wellformedness preprocessing step as the >>> other calls). >>> -Aaron >>> >>> >>> Jonathan Ostwald wrote: >>>> NDR-test does not allow escaped markup (e.g., "<") in the >>>> inputXML parameter. >>>> >>>> Is this intentional? >>>> >>>> thanks, >>>> >>>> Jonathan >>>> >>>> ----------------------- >>>> >>>> >>> xmlns="http://ns.nsdl.org/ndr/request_v1.00/" >>>> xsi:schemaLocation="http://ns.nsdl.org/ndr/request_v1.00/ >>>> http://ns.nsdl.org/schemas/ndr/request_v1.00.xsd" >>>> schemaVersion="1.00.000"> >>>> >>>> >>>> < >>>> >>>> >>>> >>>> >>>> >>>> =============== >>>> proxyResponse >>>> >>>> >>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" >>>> schemaVersion="1.00.000" >>>> xsi:schemaLocation="http://ns.nsdl.org/ndr/response_v1.00/ >>>> http://ns.nsdl.org/schemas/ndr/response_v1.00.xsd"> >>>> >>>> >>>> inputXML parameter is not well-formed >>>> xml. >>>> >>>> >>>> >> -- Jonathan Ostwald Digital Learning Sciences University Corporation for Atmospheric Research phone: 303-497-2661 e-mail: ostwald@ucar.edu http://www.dlese.org From ostwald at ucar.edu Thu Aug 16 12:18:59 2007 From: ostwald at ucar.edu (Jonathan Ostwald) Date: Thu Aug 16 12:19:03 2007 Subject: [Ndr-users] granting authority to change Message-ID: <46C478F3.9090105@ucar.edu> How do i grant authority to an agent (other than the creating agent) to make changes to an aggregator or metadataprovider? i tried adding an "authorizedToChange" relationship, but the resulting relationship was in the "nsdl" (rather than "auth") namespace and had no effect. thanks, Jonathan -- Jonathan Ostwald Digital Learning Sciences University Corporation for Atmospheric Research phone: 303-497-2661 e-mail: ostwald@ucar.edu http://www.dlese.org From birkland at cs.cornell.edu Thu Aug 16 12:52:20 2007 From: birkland at cs.cornell.edu (Aaron Birkland) Date: Thu Aug 16 12:52:25 2007 Subject: [Ndr-users] granting authority to change In-Reply-To: <46C478F3.9090105@ucar.edu> References: <46C478F3.9090105@ucar.edu> Message-ID: <46C480C4.3010609@cs.cornell.edu> The actual relationship is http://ns.nsdl.org/ndr/auth#authorizedToChange, so using the default namespace in the request will not work (and will be rejected, in the next version of the code coming out to ndrtest soon). You'll have to use "http://ns.nsdl.org/ndr/auth#". See, for example http://ndrtest.nsdl.org/api/get/2200/test.20070201142135123T -Aaron Jonathan Ostwald wrote: > How do i grant authority to an agent (other than the creating agent) > to make > changes to an aggregator or metadataprovider? > > i tried adding an "authorizedToChange" relationship, but the resulting > relationship was in the "nsdl" (rather than "auth") namespace and had > no effect. > > thanks, > > Jonathan > From ostwald at ucar.edu Wed Aug 29 17:56:29 2007 From: ostwald at ucar.edu (Jonathan Ostwald) Date: Wed Aug 29 17:56:32 2007 Subject: [Ndr-users] finding Agent objects with custom properties Message-ID: <46D5EB8D.5030706@ucar.edu> I am trying to add a property to selected Agent objects that I can then use to find them all with a Find query. Here is an Agent after adding the property: 2007-08-29T21:50:20Z /repository/api/get/2200/test.20070601114303740T 2200/test.20070601114303740T 2007-06-01T15:43:04.867Z 2007-08-29T21:25:53.706Z Active 2200/test.20070601114303740T ncs.nsdl.org Agent true ... the property I have added is true which goes along with the namespace declaration xmlns:ncs="http://ncs.nsdl.org" But I am unable to find this object using the following find request: true =============== proxyResponse 2007-08-29T21:38:27Z /repository/api/find Is there some reason my request is not finding the object? thanks, Jonathan -- Jonathan Ostwald Digital Learning Sciences University Corporation for Atmospheric Research phone: 303-497-2661 e-mail: ostwald@ucar.edu http://www.dlese.org