PART 6. Chapter 12. How to collect and use feedback from readers. Should you do audio or video recording of your sessions?

Transcription

1 TOOLKIT fr Making Written Material Clear and Effective SECTION 3: Methds fr testing written material with readers PART 6 Hw t cllect and use feedback frm readers Chapter 12 Shuld yu d audi r vide recrding f yur sessins? U.S. Department f Health and Human Services Centers fr Medicare & Medicaid Services

2 TOOLKIT Part 6, Chapter 12 Shuld yu d audi r vide recrding f yur sessins? Intrductin Shuld yu d audi recrding f yur sessins? Shuld yu d vide recrding f yur sessins? If yu recrd yur sessins, shuld yu transcribe them? Weighing the tradeffs and making yur decisin List f figures in this chapter: Figure 6-12-a. Shuld yu d audi recrding f yur sessins? Figure 6-12-b. Shuld yu d vide recrding f yur sessins? This dcument is the twelfth f 19 chapters in Part 6 f the Tlkit fr Making Written Material Clear and Effective. The Tlkit has 11 Parts. It was written fr the Centers fr Medicare & Medicaid Services (CMS) by Jeanne McGee, McGee & Evers Cnsulting, Inc. The guidelines and ther parts f the Tlkit reflect the views f the writer. CMS ffers this Tlkit as practical assistance t help yu make yur written material clear and effective (nt as requirements frm CMS).

3 CHAPTER 12: Shuld yu d audi r vide recrding f yur sessins? 162 Intrductin When yu are planning yur feedback sessins, yu will need t decide what apprach t use fr summarizing participants reactins t the written material. Fr many prjects, yu can d just fine by relying slely n written ntes, especially if yu take a few minutes t review and expand n them immediately after each sessin. Using a Sessin Summary Frm t write dwn highlights f the interview is als very helpful (see previus chapter fr details). While yu shuld cunt n taking ntes t summarize results frm yur feedback sessins, yu may want t cnsider supplementing yur ntes by recrding the sessins (with participants permissin, f curse). Recrdings give yu a mre cmplete recrd f feedback than ntes alne can prvide, and they can be gd tls fr sharing the results with thers and imprving yur interviewing skills. If yu are cnsidering making recrdings, there are imprtant factrs t cnsider, including hw participants might feel abut being recrded, as well as the added cst, especially if yu prduce written transcripts frm the recrdings yu make. T help yu decide whether audi r vide recrding makes sense fr yur prject, this chapter discusses the ptential benefits and csts, as well as ther issues t cnsider. Shuld yu d audi recrding f yur sessins? Figure 6-12-a belw utlines the ptential benefits f audi recrding and factrs t cnsider in making yur decisin.

4 CHAPTER 12: Shuld yu d audi r vide recrding f yur sessins? a. Shuld yu d audi recrding f yur sessins? Here are several pssible benefits f making audi recrdings: Yu can use the recrding t imprve yur written ntes and refresh yur memry. Having a recrding makes it easy t review the sessin and make additins r crrectins t yur ntes. Yu can use the recrdings t share the feedback with ther peple. When yu are analyzing and reprting yur results, yu can pull ut qutatins frm the recrdings and use them as examples, either in written frm r as audi clips. Qutes frm yur feedback sessins can be a pwerful tl fr shwing and cnvincing peple abut changes that need t be made in the written material. If yu want t, yu can create a full transcript frm the recrdings. Yu can use recrdings as tls fr training and skill develpment. Listening t recrdings f feedback sessins can help peple learn hw t cnduct feedback interviews and give them a way t practice taking ntes. If yu re an experienced interviewer, listening t recrdings can help yu hne yur skills. Here are things t cnsider: Recrding might make sme participants feel uncmfrtable r inhibited. If yu plan t recrd a sessin, yu need t get peple s permissin in advance. Yu will prbably find that mst f yur participants are willing t be audi taped, especially when yu explain hw the tape will be used. If nt, yu can skip the recrding and just rely n written ntes. Audi recrdings cannt capture the visual aspects f written materials. Whether this is a serius limitatin r nt depends n hw yu plan t use the audi recrdings: If yu are using audi recrdings fr enhancing yur ntes r analyzing yur feedback, it is ften quite limiting t have nly the audi part f the interview. If yu dn t knw what a reader is lking at, it is frustrating t listen t an audi tape f an interview and hear the reader say, that part is really cnfusing, r I wnder why they did this? Interviewers can help cmpensate fr this prblem by phrasing their wn remarks with the audi recrding in mind. Fr example, instead f saying, What s cnfusing abut

5 CHAPTER 12: Shuld yu d audi r vide recrding f yur sessins? 164 it?, they can say, What s cnfusing abut this tp part f page 3? It takes practice and finesse fr interviewers t learn t embed this type f extra infrmatin rutinely. Even when they d, there are inherent limits n hw much yu can cmpensate fr the absence f the visual dimensin in an audi recrding when yur purpse is getting feedback t smething that is visual. Fr training purpses and skill imprvement, the absence f the visual dimensin is less imprtant. When yu are cnducting a feedback sessin, yur attentin is fcused n the immediate task rather than n hw well yu are ding. Later, when yu are listening t a recrding f the sessin, yu can analyze hw yu cnducted the interview. With audi nly, yu may nt knw what smene is lking at when they make a remark, but yu will be able t hear their tne f vice and yur wn tne f vice. As yu listen t the audi, yu can assess the decisins yu made in guiding the discussin, such as when yu chse t ask a fllw-up questin and when yu didn t, and get ideas abut hw t d better next time. If equipment is readily available, it generally takes little additinal time r mney t d audi recrding f yur feedback sessins. Yu can make audi recrdings with a simple cassette tape recrder r a digital recding device. If recrding the sessins is imprtant fr yur prject and yu dn t have recrding equipment, yu can prbably buy smething suitable at relatively lw cst. If yu are ding individual interviews, a simple recrder with a decent micrphne is all yu need. If yu are ding fcus grups, the quality and placement f yur recrding equipment becmes mre imprtant, especially if the grup is large r yu hld the grup in a large rm with pr nise cntrl. Prfessinal facilities are usually already wired fr high-quality audi recrding. It takes time t put the recrdings t full use, and this may add t the expense. While there are many benefits t having audi recrdings, keep in mind that it takes sme time t use them. If yu are using audi recrdings t enhance yur written ntes, yu can use them selectively by listening t the recrdings nly when yu need t, such as when yu had truble keeping up with taking ntes. If yu decide that a partial r full written transcript f the recrdings is needed, this will add t the expense (see the sectin belw abut written transcriptin). Surce: Prepared fr use in this Tlkit, based n persnal experiences f the writer and clleagues.

6 CHAPTER 12: Shuld yu d audi r vide recrding f yur sessins? 165 Shuld yu d vide recrding f yur sessins? Figure 6-12-b belw utlines the ptential benefits f vide recrding and factrs t cnsider in making yur decisin b. Shuld yu d vide recrding f yur sessins? In sessins with readers, vide recrding is nt as widely used as audi recrding. Vide recrding tends t be mre expensive and mre cmplicated t arrange. Vide recrding is much mre likely t be included in large prjects dne by researchers than in smaller prjects dne by nn-researchers. Vide is als mre cmmnly used fr fcus grups than fr individual interviews. The ptential benefits f vide recrding a feedback sessin include all f the ptential benefits f audi recrding that we discussed in Figure 6-12-a. These are: T imprve yur written ntes and refresh yur memry. T help share the feedback with ther peple. This might include using the recrding t prduce a written transcript. As tls fr training and skill imprvement. Since vide includes the visual aspects as well as the audi, it ffers these additinal benefits: Having vide can help yu identify wh is speaking and allw yu t study nn-verbal cmmunicatin. Vide is especially useful fr analyzing grup sessins, because yu can use it t check n wh is saying what. Watching a vide may give yu insights int grup dynamics

7 CHAPTER 12: Shuld yu d audi r vide recrding f yur sessins? 166 and their impact n the discussin that yu wuldn t be able t get frm just an audi recrding r written transcript. Visual cues can enhance yur understanding f what peple say. Reviewing a vide can smetimes help yu see what they are reacting t when they cmment n the written material. If yu have vide recrdings, yu can extract vide clips t share with thers. These can be even mre cmpelling than audi clips, because peple can watch as well as listen t the feedback readers give. Here are sme things t cnsider: Cmpared t audi recrding, vide recrding is mre intrusive and mre likely t make peple feel uncmfrtable r inhibited. The added dimensin f vide tends t make privacy and dignity f greater cncern t participants. It is ne thing t have yur vice recrded, and quite anther t have what yu say and d captured n vide. If the vide camera is right nearby, it can be distracting and ptentially disturbing t participants than a tape recrder. If the camera is cncealed r munted in an unbtrusive place, as it ften is in a cmmercial facility, it will be less distracting. (Of curse, whether the camera is cncealed r nt, yu must tell the participant abut it and get permissin in advance t d the vide recrding.) Althugh vide recrding includes the visual cmpnent f the sessins, it can still be quite hard t tell which part f the written material readers are lking at when they make a remark. By definitin, vide is better than audi alne at helping yu watch hw readers react t written materials. Hwever, adding vide des nt necessarily end the frustratin f being unable t cnnect what readers say with what they are lking at. While vide usually shws the readers fairly well, it is ften nt as gd as yu might assume at shwing the materials themselves r the readers interacting with the materials. This is a significant limitatin, and there are several reasns fr it: Yu cannt cntrl what the reader des with the material. Fr example, if a reader lifts up a bklet frm the table and hlds it in frnt f her, r belw the edge f the table, yu will nt be able t film what she is lking at (unless yu have a very intrusive camera peratr which wuldn t be a gd idea in any case). It may nt be feasible t use a camera angle that can track the expressins and eye mvements f a reader wh is lking dwn at written material. Vides are seldm sht clse enugh and sharp enugh t reveal the details f the written materials as well as the reader interacting with the material. Many vides are sht frm a fixed camera psitin that captures the full scene. If yu can see all f the peple arund the table in a fcus grup, r the interviewer and the reader and the

8 CHAPTER 12: Shuld yu d audi r vide recrding f yur sessins? 167 materials, yu are nt ging t be able t see the details f the materials. If the vide zms in, yu may see the written material mre clearly, but lse yur view f the reader. Sme cmmercial facilities prvide a superir videtaping capability that is flexible enugh t address these prblems, but these facilities tend t be quite expensive, and they may prvide much mre detailed feedback than yu want r need. Fr example, there are usability testing labratries designed t capture keystrkes and eye mvements f readers wh are using websites. Vides sht thrugh a ne-way mirrr at a cmmercial facility are ften rather fuzzy. Extracting vide clips is generally time-cnsuming and cstly. While vide clips have high impact as examples, it can take an enrmus amunt f time t identify apprpriate clips, mark them fr extractin, and edit them fr final use. This level f effrt prbably makes sense nly fr a large-scale research prject. Surce: Prepared fr use in this Tlkit, based n persnal experiences f the writer and clleagues. If yu recrd yur sessins, shuld yu transcribe them? If yu recrd yur feedback sessins, yu can use the recrdings t make a transcript f the sessin. This transcript prvides a cmplete written recrd f a sessin that is easy t share with thers and easy t use in analyzing yur results. It can be useful t have a written transcriptin f what is said during sessins with readers, but it takes time and mney t prduce ne: T help estimate csts, figure that a transcriber will spend rughly three t fur times as lng as the sessin t create the written transcript. Fr example, transcribing an hur-lng sessin will take abut three t fur hurs f transcriptin labr. If yu have dne a series f individual sessins, this can add up quickly. Hw lng it takes t prduce a gd written transcript will vary depending n the quality f the recrding, the skills f the transcriber, and the quality f the equipment used fr transcriptin. If the vices are hard t hear, r there s a lt f backgrund nise, transcriptin will take lnger, because the transcriber will need t play parts f the recrding ver and ver again. Transcribers wh are skilled and experienced will tend t prduce better transcriptins. Fr example, they will insert ntes t let yu knw when the speaker was laughing r when the tne f vice was irnic r sarcastic. Withut these ntes frm the transcriber, yu wuld

9 CHAPTER 12: Shuld yu d audi r vide recrding f yur sessins? 168 misinterpret the meaning f what was said, because yu can t hear the tne f vice when yu read a written transcript. Befre yu make this investment in transcriptin, think abut hw yu wuld actually use the transcript, and whether it s really necessary t have transcripts fr yur prject. Keep in mind that fr many purpses, yu can put an audi r vide recrding t gd use withut transcribing it. Weighing the tradeffs and making yur decisin Fr yur prject, des it make sense t rely slely n written ntes, r shuld yu include sme type f recrding as well? Audi r vide recrdings can t substitute fr careful, thrugh nte taking, but they can be useful fr many purpses. Yu will need t weigh the trade-ffs f whether t recrd r nt based n the needs f yur prject, the nature f yur written materials and participants, and what yu can affrd in terms f time and resurces. If yu decide t make the investment in audi r vide recrding, be sure that yu have a specific plan fr hw yu will put yur recrdings t use.

10 CHAPTER 12: Shuld yu d audi r vide recrding f yur sessins? T view, save, r print all r parts f this Tlkit frm yur persnal cmputer, visit and select Outreach & Educatin. CMS Prduct N September 2010