Evaluating the Effect of Justification and Confidence Information on User Perception of a Privacy Policy Summarization Tool.

摘要: Privacy policies are long and cumbersome for users to read. To support understanding of the information contained in privacy policies, automated analysis of textual data can be used to obtain a summary of their content, which can then be presented in a shorter, more usable format. However, these tools are not perfect and users indicate concern about the trustworthiness of their results. Although some of these tools provide information about their performance, the effect if this information has not been investigated. In order to address this, we conducted an experimental study to evaluate whether providing explanatory information such as result confidence and justification influences users’s understanding of the privacy policy content and perception of the tool. The results suggest that presenting a justification of the results, in the form of a policy fragment, can increase intention to use the tool and improve perception of trustworthiness and usefulness. On the other hand, showing only a result confidence percentage did not improve perception of the tool, nor did it help to communicate the possibility of incorrect results. We discuss these results and their implications for the design of privacy policy summarization tools.