Log in

We feature members, external experts, and other great people, so check back regularly for new posts! Have something to share? Contact our Communications Chair!

  • October 01, 2018 5:31 PM | Deven Wisner (Administrator)


    Do you want to make a positive social impact with your work? 

    When you work with clients, when you present your research...why do you do it? 

    Is it just a CV-builder? If so, then this probably isn’t for you :) But my guess is that it's because you actually want people to be inspired and motivated enough to use that information for social good. 

    When you present about the strengths of a program or policyI'm guessing it's because you your clients to keep those elements. 

    When you present about the weaknesses of a program or policy, I'm guessing it's because you want them to implement changes to improve those things.

    You get the point. 

    But what if I told you that you'll probably never achieve that goal if you're presenting in the standard way. 

    You've heard of the "Standard American Diet" (SAD) right? Well I'm calling this the "Standard Academic Delivery" (also SAD). 

    Here's what the SAD looks like:

    • Slides that consistently have 50+ words per slide.
    • Presentations that follow the 1 minute per slide rule.
    • Presentations with no storytelling, passion, excitement, or enthusiasm. 
    • Overwhelming presentations that have too much information and not enough audience engagement. 
    • Presentations that aren't well-rehearsed. 
    • Presenters who say things like, "Oh that's a lot of text, I'm not going to read all that." *next slide*
    • Dataviz that's just graphs/charts copied & pasted from a pub. 
    • No visuals. 
    • Visuals that are word clouds, puzzle pieces, hands shaking, or bubble men (WTF ARE THOSE OMG).
    • Presentations where the main "visual" is a slide template

    Evaluators aren’t immune from this just because they aren’t (always) “academics.”

    So how can you prevent yourself from following the SAD? It is, after all, standard...which means it's the status quo. That's both good news and bad news. 

    The bad news is that is there's a lot of pressure and nay-saying to keep people following the status quo. Any time you're going against the status quo, you will have people trying to convince you to "get back in line." Just remember: That's usually because they don't want the bar raised and they don't want to be required to put more effort into their presentations. When they tell you that effective, visual presentations are unprofessional, they're wrong. What's unprofessional about creating presentations that inspire people to act? That increase the retention of scientific or academic information? That increase the accessibility of complex info? That's exactly what professionalism is to me. 

    Look. There's a reason #DeathByPowerpoint is the SAD--it's easy. 

    The good news is that most presentations are boring, ineffective, and unengaging. If you get started on creating better presentations now, then you actually don't have to do all that much to create presentations that stand out for being awesome! You can learn the basics and stand out, and then increase your skills from there to be even more awesome.

    Don't wait until most people are creating awesome presentations because then you'll be scrambling to catch up. 

    This is something that even hummingbirds can demonstrate. Let's check in on my friend, the sword-billed hummingbird, so I can show you what I mean. 

    Please keep in mind that I'm not a scientist who studies birds, so this is a highly simplified summary with a sprinkle of Echo Flair :) 

    OK, so here's the setup.  

    We're in South America, and there are lots of flowers and pretty hummingbirds here.


    Which is kinda cool, we have a nice community of hummingbirds here...except there's also a lot of competition for food. A single flower's nectar only goes so far, amirite? 


    That was like, super stressful, for some hummingbirds at one point. I can relate to that. It's kinda like being anti-capitalist in a capitalist society. I hate competing with folks. I'd rather we just, I dunno, collaborated and sh*t to end all these problems. But hey. I'm just one person.

    Anyway. Then evolution happened and a new hummingbird came along: the sword-billed hummingbird. Unlike these standard hummingbirds, this one has a really long bill. It probably didn't want to mutate and be different. It probably just wanted more flowers to be available, but hey.Evolution, amirite?

    So I want you to guess: how do you think that turns into an advantage for this hummingbird? 

    If you guessed that it can essentially cut in line to reach that flower first, you'd be ...


    wrong! 

    Their long bill let's them bypass the whole competitive environment altogether. 

    Their long bill let's them access whole new--really long--flowers, like those in the Passiflora and Datura family. 


    As a result, this hummingbird effectively has no competition (from other hummingbirds) for food. 

    Think about it. That's a less stressful existence for this hummingbird. It can float around the flowers, bypassing the crowd, and achieve its food goals without worrying about others. 

    I want you to have that same advantage when you're presenting your work. 

    The hummingbirds are born the way they are--they have no choice to have either a short bill or a long bill. Luckily, you have a choice when it comes to designing ineffective presentations or engaging presentations. 

    Right now, the SAD is ineffective presentations. But, there's nothing to say that pretty soon the SAD will be effective, engaging, awesome presentations. In fact, I'm trying to change the SAD from #DeathByPowerpoint to #InspiredByPowerpoint. 

    So now I ask you: Would you rather be part of this movement with me? Or, would you rather be sucker punched in 5 years and realize that now your slides stand out for being ineffective :( 

    Clearly, I want you to join me. But here's some more illustrations to make my point even clearer (hopefully). 

    An Illustrated Story of the Standard Academic Delivery (SAD)

    People using the SAD start with slide templates. 


    And then type their entire presentation and read from their slides to their audience, apologizing for this (but still doing it every time). When they realize they're running out of time they start saying things like, "Oh that's a lot of text, I'm not going to read that to you" ... and then skip a few slides.

    Pretty soon, a pattern emerges across presenters. All templates, almost all text, no good storyboard or storytelling, and no inspiration. The audience is bored, stares at their phone, wondering why these always feel like a waste of time. 

    Not a single audience member thinks, "Well, at least they're being professional!" 


    But then a breath of fresh air comes along.

    Someone who actually practiced the presentation beforehand and delivers it with enthusiasm. Someone who doesn't use a template, uses effective information design principles, and uses the slides as a visual complement rather than as their speaker notes. 

    The audience is actually interested, engaged, and checking their phone approximately 50% less this time. They feel like they learned something new, feel empowered to act, and actually want to get to know this presenter more and make a note to network with them later. 


    Which one do you want for you?

    I know which one I want for you :)


    Want more tips? I’ve made a free download available that has more effective presenting tips. You can down download those presentation tips here.  

    About the author

    Hi! I’m Dr. Echo Rivera, owner of Creative Research Communications, LLC via echorivera.com. My passion is helping researchers, academics, scientists, and evaluators become effective visual communicators. I love training folks on how to create astronomically awesome slide presentations for lectures, conferences, and workshops. I also love to draw comics and want to see more comics used in research, evaluation, and teaching. I’d love to connect with you on twitter, YouTube, Facebook, Instagram, or LinkedIn.

    psst...this blog includes copyrighted illustrations I made. You can share this post widely, but if you want to use these, email me first!

    This blog post is an adapted version of an article that originally appeared on Echo's blog


  • August 29, 2018 7:33 AM | Deven Wisner (Administrator)

    Rakesh Mohan has graciously agreed to share his insight on leveraging outside consultants to increase the evaluation capacity of his office. Rakesh has been the director of the Office of Performance Evaluations (OPE) since 2002. His office is an independent, nonpartisan agency of the Idaho State Legislature. OPE is a small shop of six evaluators, an administrative coordinator, and the director. The office was recognized twice by the American Evaluation Association (AEA):

    • 2016 Outstanding Evaluation Award
    • 2011 Alva and Gunner Myrdal Government Evaluation Award

    Small evaluation shops shouldn’t shy away from taking on large, complex studies. They can tackle these studies with confidence and produce useful evaluations by strategically contracting with consultants.

    In 2008, the Idaho legislature asked my office to conduct an evaluation of the state’s transportation department. My staff and I did not know anything about construction and maintenance of highways and bridges. We had no experience in conducting a complex, or even a simple, transportation study. Despite this limitation, I agreed to undertake the study.

    Why did I tell the legislature that my office could do this study? Being fully aware of the risk of failure and possibility of losing my job, I agreed to do the study for two reasons: (1) I wanted to demonstrate to the legislature that my office can deliver large, complex evaluations and be responsive to its information needs, and (2) I had confidence that I could assemble a team of subject matter experts and experienced evaluators to do such a study.

    I worked closely with key legislators and told them about my plans. They placed their trust in me and appropriated a significant sum of $550,000 to conduct the study. This amount was in addition to my annual appropriation of $822,200 to run the office and conduct other studies during the year.

    I contracted with 11 consultants as one team and three individual consultants. Together these 14 consultants completed the evaluation on time in six months and 20% under budget. I managed the entire evaluation—from designing the scope to issuing the final report. The legislature and the governor used the report to make policy and budget decisions. The transportation department used the report to improve its operations.

    Over the years I have used consultants for three main reasons: (1) acquire subject matter expertise for evaluations, (2) conduct quality control reviews before evaluations are publicly released, and (3) provide customized training for my staff.

    Subject Matter Expertise. The transportation study shows how consultants can be used to conduct an entire study. My office also uses consultants as subject matter experts for specific areas within a study. These consultants work as team members with my staff. Recent examples of studies where we used consultants as team members include a wide range of topics:

    Representation for Children and Youth in Child Protection Cases

    State Jurisdiction in Indian Country

    Child Welfare System

    K–12 Education Funding

    Risk of Bias in Administrative Hearings

    Challenges and Approaches to Meeting Water Quality Standards

    Quality Control. Before I migrated to evaluation work, I had been trained as a performance auditor. Audit standards require that audit shops go through a peer review once every three years. The purpose of a peer review is to assure that audit shops comply with audit standards and adhere to their audit policies and procedures. I found one critical limitation with this type of quality control—it is conducted after the audits become public. I believe a better peer review process would be the one that ensures the quality of an evaluation before it becomes public.

    I began to use consultants for selective quality control work on our evaluations. This external review is in addition to the quality control review that evaluators perform in-house. Consultant reviews vary depending on project needs; they can take as few as 10 hours to as many as 100 or more hours. Below is a generic example of the scope of work for a consultant:

    Quality Control Review Tasks

    The contractor agrees to conduct a quality control review and communicate her/his comments to the director and team lead via email, phone, or other means as necessary. Specifically, the contractor will do the following:

    1

    Review the project scope.

    2

    Review the initial project plan and any significant subsequent changes to the initial plan (this includes reviewing methodologies used to address study objectives).

    3

    Review key workpapers involving complex analyses (both qualitative and quantitative). Determine whether methodology used in the analysis was appropriate, the analysis was performed accurately, and the conclusions drawn from the analysis were logical.

    4

    Participate in a report message meeting with the team.

    5

    Review the report message write-up (the report write-up should clearly identify the main message based on the key findings).

    6

    Review the report draft for logic, flow, and accuracy based on evaluators’ work using the following criteria:

    (a)   Whether the report is logical

    (b)  Whether the methodology is sound

    (c)   Whether the conclusions and recommendations are adequately supported

    (d)  Whether the tone of the report is balanced

    (e)   Additional ideas to improve the quality of the report

    7

    Verify cross-referencing between the report draft and workpapers for accuracy by checking selected portions of the report.

    8

    Review the report’s one-page highlights and press release for accuracy, readability, and tone.

    Professional Standards

    All work will be conducted in accordance with one of the following sets of standards:

    • Generally Accepted Government Auditing Standards, US Government Accountability Office
    • American Evaluation Association’s Guiding Principles for Evaluators
    • Program Evaluation Standards, the Joint Committee on Standards for Educational Evaluation


    Customized Training. I invite consultants to give my staff customized training. These training sessions have covered different types of analyses such as qualitative, quantitative, cost-benefit, and staffing and workload. Other trainings have included data visualization, leadership, and working in public policy environments. Trainings have lasted from two hours to two days. They have an added benefit—I can assess whether the trainer could be a good consultant for future evaluations.

    There is no substitute for having a highly trained and motivated staff—and I’m fortunate to have such staff. By complementing them with the use of consultants, I have been able to take my staff to greater heights of evaluation capacity and excellence. I believe this approach could work well for any evaluation shop that has limited capacity to conduct large, complex studies. 

    -Rakesh Mohan, Office of Performance Evaluations (OPE)


  • July 24, 2018 10:40 AM | Deven Wisner (Administrator)

    Hello! I’m Sheila B. Robinson, Ed.D of Custom Professional Learning, LLC. I’m an educator, consultant, and program evaluator with a passion for professional learning. I design and facilitate professional development courses on program evaluation, survey design, data visualization, and presentation design. My book, Designing Quality Survey Questions, was published by Sage Publications in 2018.

    I’m an active American Evaluation Association member and currently Lead Curator and content writer for AEA365 Tip-A-Day By and For Evaluators, their daily blog on program evaluation, and Coordinator of the Potent Presentations Initiative (p2i). I’m Chair of both AEA’s Professional Development Workgroup, and Awards Committee. I’m also past Chair and Program Chair of the Pre-K12 Educational Evaluation Topical Interest Group (TIG).

    I have taught graduate courses on program evaluation and professional development design and evaluation at the University of Rochester’s Warner School of Education where I received my doctorate in Educational Leadership and Program Evaluation Certificate. 

    The Potent Presentations Initiative (p2i) is sponsored by the American Evaluation Association and exists for the explicit purpose of helping evaluators improve their presentation skills, both at conferences and in individual evaluation practice. Potent Presenters think about three key components of a compelling presentation: Message, Design, and Delivery, and our free resources are largely organized around these areas.

    P2i got its start back in 2012 with Stephanie Evergreen, data visualization and presentations expert, and has continued to grow over the years, with several people contributing to the effort, including me! Initial identification of the three key components and inspiration for much of the early content and came from a study of a dozen highly regarded AEA presenters dubbed the “Dynamic Dozen,” who had the highest feedback scores from pre-conference professional development and Summer Institute workshops, and Coffee Break Webinars.

    Our p2i website features a p2i Presentation Tools & Guidelines page with free, downloadable checklists and worksheets, along with webinars and slides on Message, Design, and Delivery. Check out the Poster Presentations page for additional guidance and tools for posters. And our Presentation Preparation Checklist is a perennial favorite that guides you along a well-paced schedule to a prepared and calm session. Our newest resources are the Audience Engagement Strategy Book and Guidelines for Handouts tool, my contributions to the p2i collection. Up next will be a resource dedicated to helping you use images in presentations.

    As current p2i coordinator, I’m always on the lookout for ideas for new content, and engage with an advisory team a few times per year for input. In a recent effort to catalyze ideas I asked evaluators to participate in a brief, informal survey about presenting to help inform our future work. The survey asked about people’s past and future presentation work and what might help them improve their practice. I shared the link to “You, the Presenter: What Would Help You Up Your Game?” in an aea365 post and on EvalTalk, and 190 people responded! The results are quite informative, and I’ve been sharing them monthly in my p2i column in the AEA monthly newsletter.

    Are you getting ready to submit a proposal to present at a conference? Have you been hired by an organization to give a presentation? Or, do you have an upcoming presentation you need to make to your colleagues? It’s a great time to check out AEA’s Potent Presentations Initiative and download free resources to supercharge your next presentation!

    -Sheila B. Robinson, Ed.D., Custom Professional Learning, LLC



  • May 23, 2018 10:38 AM | Deven Wisner (Administrator)

    Happy monsoon season to all! This time of year is always a treat following a blistering couple of months. The AZENet Board of Directors is busy this summer diving into our 2018-2019 strategic plan for the Network. As a reminder, our work over this next year aligns with three key priorities: membership, sustainability, and community embeddedness. I welcome Network members to get involved to support the plan themselves in a few key ways.

    Invite a colleague to your next AZENet membership event to get to know the network. Pass on professional development activities on your listserv or contact list. We welcome all evaluative types – not just those with the word “evaluator” in their name. Know an organization that does evaluative work who should be engaged in this Network? Share our website or contact the Membership Recruitment Chair to introduce them to us.

    Consider joining our membership committee to expand your network and help us with outreach and recruitment. Contact Membership Recruitment Chair, Nicole Janich at Membership@azenet.org for more information and to get involved.

    A key focus this year will be to connect with the next generation of evaluation professionals in post-secondary institutions in the state. Are you interested in participating in mentoring events for students and early career evaluators? The Network is also exploring opportunities to connect directly with our Arizona communities to provide professional development & outreach.

    Interested in sharing your skills in a professional development workshop or have an idea for a community event? Contact the Professional Development Chair, Roseanne Schuster, at ProfessionalDevelopment@azenet.org.

    I invite you to connect with us to grow your connections to evaluation in Arizona. Have a thought or comment? Email me directly at President@azenet.org.

    In Evaluation,

    Jenny McCullough Cosgrove, AZENet President/CEO


Powered by Wild Apricot Membership Software