Re: [pki-tc] Gearing up for PKI Survey Promotion

[Stephen Wilson <swilson@lockstep.com.au>]

Arshad, 

Many many thanks for the excellent feedback.  You raise some good points,
better late than never!  My thoughts in line below ... 

> Stephen/Ann,
> 
> Some feedback on the survey I received and tried to fill out today:
> 
> 1) I am not a Microsoft Office user, so the fact that the
>     survey is distributed in Excel format is a problem for me,
>     as OpenOffice 2.0 does not faithfully reproduce the format
>     you carefully created.  Additionally, while OpenOffice will
>     save the document in Excel format, I'm not sure how it will
>     look when you open it up in Excel again.  And no, unfortunately,
>     I don't have a copy of Excel even in my lab Windows machines.
> 
>     While I understand that most of the world uses MS-Office,
>     OASIS has gotten behind the OpenDocument standard, and as
>     a subcommitee, I think we should support it by providing the
>     survey in an OpenDocument format document too.  I would
>     encourage you to download OpenOffice 2.0 and add the
>     OpenDocument version too.

I tend to agree that in principle -- and it's an important principle --
that OASIS exercises should utilise OpenDocument.  Having said that, like
you I have a practical problem, in that I have no familiarity with the
standard, nor OpenOffice.  It falls outside my brief at this stage to do
the document translation, unless it is truly trivial.  But if it really is
trivial, can we perhaps get the new OASIS TC Admin person to take care of
this?  Ann, you might have missed discussion of this person early on the TC
con call yesterday, but they are due to start in Feb, with 5 hours a week 
available to us. 

> 2) Steve Hanna's e-mail address has changed, as you might
>     know; might be better to get OASIS to give him a generic
>     chair-pkitc@oasis-open.org e-mail address.

Thanks, good catch! 

> 3) The survey should recognize that consulting companies (like
>     StrongAuth) build many PKIs, and even though they may be small
>     (5 people in our case), they may have built PKIs for companies
>     of much larger size (120,000 employees in the case of one of
>     our customers).  Currently, the survey does not explain how one
>     should address this.

Good point.  I think we should ask for the size (and other details) of the
organisation that is sponsoring the PKI, not building it.  I will clarify
the instructions. 

> 4) In the question "Obstacles to PKI" it is not clear whether
>     I should allot 10 points per question, or across all the 16
>     questions.  Is this meant to be a scale of 1 to 10, with 10
>     being "most important" or "least important" to me?  Its not
>     clear how I should be responding.

It's per question.  

The methodology is the same as that used in the first two surveys.  But I
may not have used the same form of words in the instructions.  Steve: do
you know if people were OK with the points allocation scheme last time?  

> 5) Same observation as (4) for questions on "Potential Improvements
>     to Software", "Problematic Costs", "Understanding" and
>     "Interoperability".
> 
> 6) I'm not sure how valid my opinions would be on the "Quantitative
>     and "Qualitative" questions.  I am a supplier of PKI services, thus
>     making me biased towards the technology, and I'm an active OASIS
>     member, so I may be too close to the problem.
>     I've left them blank for the time being; let me know how you'd
>     like me to respond to these sections.

I trust you to be sensible in your responses!  You wouldn't be spending
some much time on the PKI TC if you didn't have a realistic view!  

> 7) To make things worse, after spending 45 minutes on the survey,
>     the application locked up as I got to the section on the projects.
>     Not sure if it had to do with the Excel spreadsheet in OpenOffice
>     or not, but I had to kill it and lost all the work I'd done so far.
>     (I'll wait for your responses before trying again).

I am so sorry to hear that Arshad! 

> 8) It might be useful to break down the question on "Costs too high"
>     into Capital, Implementation and Operational costs and determine
>     which costs people are worried about.

I am not exactly sure which question you mean, but regardless, I think the
wording has been carried over from the past surveys, and we did this in
some cases in order to help make trend comparisons. 

> 9) WRT "Enrollment too complicated", it might be useful to break
>     this question into the Technical process and the Business process
>     and determine which part do respondents believe is too complex.

Same point as (8). 

> 10) WRT "Too much legal work required", it might be helpful to ask
>      the question why respondents think there should be legalities
>      involved with a PKI when there aren't any with User ID/Passwords;
>      and US Federal law allows for people to assent to contracts with
>      an "X" in a checkbox on electronic forms.  My point is, shouldn't
>      IT organizations be more worried about data-integrity and security
>      rather than legal-enforcement of the digital signature?  Is this
>      one of the reasons PKI is too complex?

Same point as (8) ... but I agree we should try to extract more information
somehow on this point.  I agree totally that the legal complexities need
not be much different from other technologies.  I have my own pet thoeires
wht this has come to be so.  I think the presence of a high proportion of
IT lawyers or law-qualified technologists in the early working groups was a
big factor plus the preeminence of the ABA.  Thus RFC 2527 and RFC 3647 are
chock full of legal issues like IP rights and liability provisions etc.  In
my view these issues are badly misplaced in most any CPS.  

But perhaps the deepest factor is that very elaborate PKI processes were
put in place (in X.509 v3, RFC 2527, Webtrust for CAs etc) well before any
major applications were up and running.  This meant that risk management
strategies were being codified and standardised around enrolment processes
(identification) without having any idea what certificateswere being use
for.  How on earth is such an exercise even possible?  Is there any
precedent anywhere else for risk management methods to be established
without reference to the application?    In Australia, there are deep
expectations that any digital certificate subject has to be registerer
using passport-strength proof of ID, regardless of application.  Another
contributer to this effect is the impression that most people have of PKI
being expressly an identification technology.  

And a nice counter example highlighting how other technologies are treated
more simply ... in Australia, as I am sure in the US, several airlines
offer self service kiosks for printing your own boarding passes.  You swipe
your airline member card, or your credit card, and they match your name
from the mag stripe track against the book record.  Nice and simple.  It's
been around for about four years.  BUT I bet you anything that if the core
security technology was digital certificates, we would STILL be debating
the legal liabilities! 
 
> 11) The "Implementation" and "Cost Detail" sheets are over-whelming,
>      Stephen.  I doubt that any IT organization has that level of
>      detail - or would be willing to part with that much information
>      even if they did.

It's a shame to be debating this point now.  The main theme of the third
survey is to get detailed cost and benefit data.  We've been developing
these sheets for a long time now, and they are based on the successful
Supply Chain model we developed in the Education SC ROI project.  I agree
they're possibly overwhelming, but what else are we to do to get this data?  

>      This may be the biggest barrier to getting quality responses to
>      the survey.  While I don't deny the usefulness of such information
>      for an analysis, I doubt we'll get that level of clarity in the
>      responses.  Even though StrongAuth has implemented 4 PKI's in the
>      last 4-5 years, even I don't have privy to all the information in
>      that survey; and I can guarantee that the people I know in those
>      companies will not have all the answers either.

I had thought for a long time that these sheets are best used in a face to
face interview situation.  But frankly I have stopped pushing for
interviews because the response from the TC at large to participate has
been so underwhelming.  We will need a committed team of interviewers to
spend several days in total at conferences to gather the data, and I just
don't see that happening on present indications.  I cannot even get TC
members to advance the names of target respondents, despite probably three
requests for suggestions. 

An alternative might be to make interviewing part of the brief for the next
stage of paid consultancy.  This would allow us to undertake a predictable
number of interviews and to put serious effort into it, not feasible for
volunteers to do.  Ann, I could factor that into the next proposal? 

> I hate to be doing this at the stage where you're getting ready to
> announce and release the survey, but is there any way you can cut
> this back to half its size?

We could certainly make the cost/benefit sheets optional.  We cannot afford
to put people off!  But we need to get the numerical data somehow, and I
think that there are so many architectures for building PKI now that unless
we get people to break costs down as per the supply chain model, we will
just get big numbers that cannot be compared with each other.  It is said
for example that the USPTO spent around US$2,000,000 on its PKI.  But the
really interesting thing is how was that money spent?  How much was on
coding and SI versus certificate services. 

> This survey will definitely take more than an hour to fill.  Just
> collecting the information could take days, and then it might take a
> couple of hours at least to get all that information filled out.  For
> most US-based ITSec folks, this will just sit on this list of "to-do"
> things and will likely not get done.  Motivated people (as on the TC)
> will help to get this moving forward, but most other people are
> probably swamped with day-to-day things to provide this much
> information.
> 
> I apologize for not having been involved earlier and this late feedback,
> but I think its more important to be honest with you and do the right
> thing than to "just try to do the survey" and ignore the fact that this
> survey will be daunting to most respondents.

Thanks Arshad, apologies accepted!  It is certainly better to get this
feedback now, rather than out of the survey itself when released to the public!

Ann, we need to conference on how to take it forward. 

Cheers, 

Stephen.


> Arshad Noor
> StrongAuth, Inc.

--
<Put email footer here>