Lauren Fitzharris: Good afternoon, everyone. Thank you for joining us on the June Ctsa. Program webinar
Lauren Fitzharris: for our agenda. Today we will start with ncats updates presented by Erica Roseman, followed by Ccos updates, followed by Carrie. And we have 3 working groups providing updates to the Ctsa today. Starting with translational impacts.
Lauren Fitzharris: the translational science, competency-based assessment working group and the engaging individuals with disabilities in research process and
Lauren Fitzharris: we will go ahead and get started with our 1st presenter, and I will pass it over to Erica.
Erica Rosemond: Thanks, Lauren.
Erica Rosemond: Good afternoon, or Good morning, everyone. Hopefully. Everyone's staying cool, I know, in Washington, DC. Here it is super hot
Erica Rosemond: in many different ways.
Erica Rosemond: So next slide you may have been hearing about this if you click a couple of times, Lauren, to get the red boxes up.
Erica Rosemond: This is the fiscal year 26 President's budget. In brief.
Erica Rosemond: This is a suggestion to Congress. But you might have been hearing about the reorganization
Erica Rosemond: of the Nih. The President's budget does support that into an 8 institute structure ncats would be collapsed into the National Institute of General Medical Sciences.
Erica Rosemond: The Ctsa line item remains in the President's budget, though it does have a decrease that is comparable to Nih's overall budget. Decrease in this President's budget, and you will notice that the Nih budget in this is 27 billion
Erica Rosemond: next slide.
Erica Rosemond: I wanted to give you an update on the reorganization of our review panels as a reminder. All of the review panels at the Nih have been consolidated under the center for scientific review.
Erica Rosemond: So we were successful in having all of our applications. Still be somewhat reviewed together.
Erica Rosemond: and we now have one.
Erica Rosemond: All of the panels are scheduled now, but only one I can communicate that is publicly available right now. July 21, st and that's our Rc. 2 S. The Ccias and the Ro. 3 s. Are all together. All of our one s. Are together, we have our K. 12 s. And our T. 32 s. Are together with some other similar grants.
Erica Rosemond: and the R. 25 S. Are placed in a Nih wide, R. 25 panel.
Erica Rosemond: I will communicate that program. Staff have been in close communication and collaboration with the scientific review officers assigned to these panels. They also have been
Erica Rosemond: reassigned from the Nih Institutes, but they are happy to work with us. They are not the review officers from ncats. But we are working very closely with them. And it's a really nice relationship.
Erica Rosemond: Okay, next slide.
Erica Rosemond: hopefully, many of you joined us on June second, we had a national clinical cohort collaborative Webinar, where we introduced the new vision for N. 3 C. We have communicated this on the Ctsa program webinar as well. But the recording is now, and the slides are now on our the Ncats website. If you are interested in joining
Erica Rosemond: and learning more about the new vision and being part of a domain team, please do send your email along to Ncats and n 3 c. clinical@nih.gov.
Erica Rosemond: and also, we've just recently updated a lot of our n 3 c. Web pages. So please do check them out.
Erica Rosemond: The next slide is a recent publication from the trial innovation network. It's a really nice article going over the initial consultation process and the types of trial innovation resources that are provided and no surprise. The high investigator satisfaction with these services. So please do check them out.
Erica Rosemond: And also what we heard about on Monday from the steering committee trial Innovation Network Rep. There is going to be another survey being deployed by the end of June from the tin to the Hub liaison pocs to gather information on the utilization of the recruitment Innovation center resources during the current and previous grant periods
Erica Rosemond: and access potential new areas of innovative resource development. So be on the lookout for that and please participate
Erica Rosemond: the next slide. We want to communicate that smart Irb version 3.0 is alive and well. Some of you have seen that there was a stop work order through the contract
Erica Rosemond: for smart irb that is now lifted and everything is working and functional again. So we we launched Version 3.0 on March 17, th
Erica Rosemond: and we have over 1,300 smart irb participating institutions. And we have over 430 signatory institutions. So please please go check out the website and be a part of version 3.0.
Erica Rosemond: The next slide just goes into how much 3.0 has grown, or in the 100 days.
Erica Rosemond: Okay, the next slide.
Erica Rosemond: This is an interesting notice. It is a rescission of a notice that was issued by the Nih
Erica Rosemond: a couple of months ago to change a term and condition, or to add a term and condition. So this was having to do with defining the terms of Dei and Deia for grants, and indicating that recipients that accepted grant awards
Erica Rosemond: would not operate any programs that advance or promote Dei or Deia. This notice and this term and condition of award has now been rescinded.
Erica Rosemond: So please be aware of that.
Erica Rosemond: And with that the next Ctsa program Webinar, will come to you in July.
Erica Rosemond: and I'll turn it back over to Lauren.
Erica Rosemond: Thank you.
Lauren Fitzharris: Thank you, Erica, for those updates.
Lauren Fitzharris: We'll move on in our agenda to Ccos updates. So I will pass it over to Carrie.
Kerry James: Thank you, Lauren. You could go to the next slide.
Kerry James: So we have a few announcements about new features on the Ccos website. There is a member Directory that is pretty comprehensive. You can search by group, you can search by institution. It's in the resources section of the website. And hopefully, more folks will join the website and
Kerry James: and get their their name added to the Member Directory. We did bring over a lot of the members who do not have website accounts yet, but who participate in Ctsa groups. So your name will be listed there as well.
Kerry James: Next slide.
Kerry James: And there is also a global site search on the website. Now, you can find it up in the upper right hand corner of the landing page. You can search by keyword more details on the website. But I encourage you to test it out next slide.
Kerry James: Ccos. It's sort of related to what I said before, we are going to require Ccos accounts for new members for anybody who wants to join a group starting on September first.st This is going to be a requirement because we're building a workflow that you can join the group or actually leave a group as well from your profile on the Ccos website. So this will apply to new group members.
Kerry James: the current members who do not have Ccos accounts. We're going to be reaching out to you to try to connect you with a Ccass account and encourage you to register as well
Kerry James: more details on that, URL, or you can get to the link to join or to register for a Ccos account through the URL here, and what you get access to is the Member Directory. All the meeting materials and archives, different types of submission forms like a poster submission, form a working group proposals. You know, there's a bunch of things that you need to that you would need an account to get access to. So I we encourage you to please sign up
Kerry James: next slide, and I feel like this slide should have some, you know.
Kerry James: I don't know. Bells and whistles with it to announce that the bids Ec. Charter has been approved by the steering committee as of June 9.th If folks want to take a peek at the bids Ec. Charter. The link is in the chat.
Kerry James: One. I wanted to reiterate a couple details about the Bids Ec. Membership. This is open to anybody who bids Ec membership is open to anybody in the hub from a hub, and they are encouraging people in the bids disciplines to to join, especially if you're in a leadership role
Kerry James: the voting members. One of the differences between the bids Charter and other Ecs is that the bids is allowed
Kerry James: 2 voting members per hub due to the size and the increase in size and scope for that Ec. And the fact that it's covering a few disciplines. So we increased the number of voting members.
Kerry James: and so voting members from different disciplines per hub are encouraged to reach out to your pis or administrators, because the selection of voting members for the bids. Ec. Is coming up. It's going to be in September. I will give you more details in just a minute.
Kerry James: So next slide, please.
Kerry James: So congratulations to the bids lead team and all the people who contributed to that. That was a big effort, and their work was really appreciated. So as far as just. This is a review. Folks, I think, know this, that
Kerry James: membership on at least one of the 4 Enterprise Committees is required per hub, but each hub is allowed to have one voting member per Ec. Except for biz, which I just mentioned, is allowed to. And so what happens annually, and we are coming up to this in September is hub pis, and administrators will confirm or select new voting members to represent their hub in the Ec.
Kerry James: Hub. Administrators have the authority and permissions on the Ccos website to make these updates, it's all automated again, it's a workflow we've built into the Ccos website. So for administrators more details to come on doing this on the website. But for the rest of the folks we have a timeline on the next slide.
Kerry James: So in September. This was. This is when we will open the sort of confirmation feature on the Ccos website for administrators to go and make changes to their Ec voting members. This is a once a year sort of period where we allow changes to be made, and this is done on purpose, because then in October is when voting members get to submit new lead team nominations
Kerry James: for the Ec. So that will take place in October, and then in November nominees will be contacted, they will be given an option to accept their nomination. Provide a bio, and then ballots will be circulated. Votes will be tallied by the end of November, and all of the announcements and lead team members will be.
Kerry James: you know, contacted in in December, and we will see. Cost will try to schedule a meeting with new chairs as schedules allow before the end of the year, so we can hit the ground running in January with the new lead teams
Kerry James: next slide, please.
Kerry James: and I know a lot of people have been interested in hearing about the fall program meeting fall program meeting planning has begun. We've had at least 3 meetings, I believe, of the Fall planning committee. They have selected a theme. You can see it here. Ctsa is leading the way. Resilience, innovation and impact. The tentative date is October 20, second, through the 24th in DC.
Kerry James: And here are the. This is the schedule day. One will be all the administrative meetings. The steering committee, the joint Ec. Leads will have an hour and a half or a 90 min meeting. The Hub administrators are going to be meeting in the afternoon for 6 h, and then 10 will also take place between 2 and 5. The Ecs
Kerry James: are on that day one as well. We needed to integrate them into the 2 and a half day period that we had spec'd out for the meeting
Kerry James: bids. We need to talk to the bids lead team to see if they want to move their session to day 3. That's possible. So we will be working with Ec. Lead teams
Kerry James: on planning these meetings in the next week. Or actually, I think we have a meeting tomorrow. So we're going to be talking about this week. Ec. Lead teams tomorrow and then day 2 is a full day. That's the 1st day of the program meeting. The poster presentations will be integrated to day 2, and then day 3 is a half day. There will be a couple sessions on day 3, and we hope to end by one o'clock
Kerry James: next slide.
Kerry James: Here are the volunteers on the Fall Planning committee. We have a nice, robust, and good representation from across the the Ctsa program.
Kerry James: and this will be on the website. You could take a peek
Kerry James: when the slides go up and other logistics updates I just wanted to share. So we have been reviewing. We've been working on logistics. We have reviewed some hotel proposals and AV proposals, and we've completed a site visit. So we have a tentative location. We're not making any announcements until we get Nih approvals. We have to have Nih approval prior to any contracting.
Kerry James: But we are going to be in the next week or so. We are going to be drafting communication to hub pis and administrators about the meeting, and I know everybody is curious about what's going to happen with attendance. And so here it is. We're allowing 9 attendees per hub. The plenary session room will allow 700 people ish, you're still going to have the 5 reps per the nofo plus one representative per Ec. So we came to 9, knowing that
Kerry James: we were going to have to increase the size of the plenary session room to accommodate all the administrative meetings on Day one.
Kerry James: And so the Fall Planning Committee, as I said, has been meeting. They're going to continue to focus on topics, format, and structure until we have the outcome of the Nih approval.
Kerry James: Next slide
Kerry James: new working groups. There were 4 proposals submitted. 2 have been approved, notifications will go out next week. The next working group proposal cycle is in September.
Kerry James: next slide.
Kerry James: and then last, but not least, and this is coming up in the engaging individuals with disabilities. Presentation by Dan Danny Armstrong. I just wanted to put an early plug that the last webinar of the Webinar series for this group is this, Friday from 2 to 3. If you are interested in this topic, please go ahead and register the zoom link will be put in the chat, and you're going to hear more about it in a few minutes.
Kerry James: That's all I have. Thanks, Lauren.
Lauren Fitzharris: Right. Thank you, Carrie.
Lauren Fitzharris: We're going to move along in our agenda, and our 1st presenter is Christy Holmes, presenting on translational impacts. Christy.
Kristi Holmes: Great. Thank you so much, and thanks for the opportunity to share an update from the working group with you. It's certainly been an exciting last few months, and there's been quite a lot accomplished, and we still have a lot of great work to come. So we look forward to sharing this with you. I co-lead the group with Dr. Emmanuel Tedda from Washington University at St. Louis's Institute for Clinical and Translational Sciences. He's an implementation scientist, and
Kristi Holmes: as many of you know, the translational sciences benefits. Model is an important framework that many of us leverage for our own evaluation within our hubs. And so that was
Kristi Holmes: born there at Washu. And so it's really wonderful to have Manny's expertise and the participation from the rest of the wash U team really actively involved. I direct informatics and data science for Northwestern Ctsa or new cats. But before that I led evaluation and continuous improvement for the Institute for about 10 years.
Kristi Holmes: And my own research interests focus on data standards, research information systems and fair practices. And so all of that really aligns synergistically with the evaluation conversation. So it's really been a pleasure to take part in this and to be able to see things move forward next slide, please.
Kristi Holmes: So as we're switching to the next slide. You can see nothing. Everyone on this call knows that nothing is possible without working together, and we really see that loud and proud in our own working group. As of last count, we have almost 140 members from 62 different Ctsa institutions.
Kristi Holmes: You can see them outlined here we have great representation across the country, but even that being said, there's always room for more. And so we eagerly encourage everyone who might want to get involved, or even just learn more to reach out, and we'd be happy to either connect you with resources or spend some time letting you know how you and other members of your hub can get involved
Kristi Holmes: next slide.
Kristi Holmes: So over the course of the last many months, we have been able to get things established, we've really been working towards specific goals. We've had 8 working group meetings. As I mentioned, we have 139 different members. It's probably a little bit more than that now, and on each call we have monthly calls. Each call has about 50 attendees.
Kristi Holmes: and while everyone is not, actively speaking, during these calls, we do have an active chat going on and we really try and create opportunities where different people can present. And we can hear different perspectives, and we'll be continuing to do that
Kristi Holmes: in the little clipboard icon you can see what a typical agenda for those meetings looks like we tend to have
Kristi Holmes: or try to have report outs about working group activities, different informational updates as needed from Ccos. We try and facilitate presentations and sharing from the Ctsa as well as from the broader impact community, and a couple of examples of recent presentations are noted below, where we got to hear a little bit more about the translational sciences benefits model.
Kristi Holmes: And we also got to hear from Dr. Catherine Graham, who's been actively involved in international impact
Kristi Holmes: work for many years about the work that she's doing at the University of Calgary next slide, please.
Kristi Holmes: So what's the point of this working group? Well, we have some specific goals, 1st and foremost, just to reemphasize the importance of collaborative work. We want to establish a community of practice.
Kristi Holmes: That community of practice is intended to not only support individual Ctsa hubs as we're working to try and understand translational impact and be able to communicate it. But we're also looking at capacity building as a consortium. So how do we think about the collaborations and the capacity that's needed in order to take some of these bigger goals forward.
Kristi Holmes: the second goal is to develop and disseminate resources and tools. So we need those tools in order to be able to accomplish the work that we want to do at the local and the broader level. We're looking to address key challenges in translation. We want to learn from each other. We want to be able to begin to
Kristi Holmes: find ways forward that look for those efficiencies that leverage technology in new ways and that really dig into this concept of meaningful impact and then finally drive future innovation and collaboration where we're strengthening the broader field and continuously improving translational impact.
Kristi Holmes: We have a terrific working group. So everybody is highly motivated, very actively involved in the group. And there are some key areas of work that are underway. So next slide, please.
Kristi Holmes: some of the accomplishments that we've had to date. So we've established the working group resources and tools. We have our regular meeting cadence. We've established 3 specific task groups which I'll talk about on the next slide. We have collaborative leadership teams for each one of those and have built those teams and are starting work.
Kristi Holmes: Even though that work is currently underway. It's never too late to get involved. And so you know, as I talk about the 3 task group areas, if you see something that's of interest, please let us know we're recruiting speakers for monthly meetings. And again, we're actively soliciting suggestions of people who we might want to hear from as well as from our own hubs. So please let us know if you'd like to join us for one of those calls.
Kristi Holmes: We've done presentations to the Ctsa evaluators to make sure that folks are aware of what's going on, and that there's that opportunity always that opportunity for Ctsa evaluators to join the group or to weigh in. And then, finally.
Kristi Holmes: we were really grateful and happy to join the evaluators at the recent acts meeting, where we developed a presentation on translational impact. So that was a great opportunity to connect in person those we do so much on zoom. But connecting in real life is always such a treat.
Kristi Holmes: Okay? Next slide.
Kristi Holmes: Okay, so here are our our 3 task groups. And these are all couched in the concept of deliverables. So the 1st of these is the impact instrument group.
Kristi Holmes: So this group is doing more than just developing an impact instrument. And they're also looking at defining key terms, developing and validating impact instruments. They're facilitating knowledge, sharing and best practices. And they've developed a logic model that is guiding the group's work. So it's really thoughtful, purposeful, stepwise approach to being able to think about
Kristi Holmes: those infrastructure resources that are needed in order to carry out impact work at the local level. I want to point out on each one of these task groups, you'll see a little leads
Kristi Holmes: at the top of the left hand side in red, and just to call out those leaders for each one of these task groups. None of this would be possible without them. So really appreciate all of their work and good ideas and passion. To to get all of this work across the finish line. Okay, next slide.
Kristi Holmes: we have an impact Summit task group. So the impact summit. What we're imagining here is we wanted to have a way to bring people together, and we've recently decided that we're going to probably host a virtual event, perhaps, over the course of 2 mornings keeping in mind time zone differences so that we can get active participation. We'll look for a wide variety of session types
Kristi Holmes: and look to engage our evaluators and hub teams in the Ctsas, and also think about other kinds of topics that we'd like to see as well. So there are
Kristi Holmes: 2 general areas like one day we might focus on Ctsa impact and highlighting that work. Maybe the second day would be more broad impact. So looking at lessons learned and ideas for the future. So this is all very much under discussion. This is what our last working group meeting was about, and we're looking forward to seeing how that evolves and grateful. We've already got a Tsbm session so really grateful to
Kristi Holmes: to the team for that suggestion. And all of the enthusiasm around that next slide.
Kristi Holmes: Our 3rd working group deliverable is an impact repository. So what do I mean by that? What I'm really looking at or or talking about is a repository or a place
Kristi Holmes: where we can deposit the items that are created as part of the working group or impact activities that are happening across the Ctsas so that they are fair. So they're made available in an easy way, so that we can learn from one another, build on each other's work that type of thing. What we're doing is we're leveraging a tool called Zenodo. Zenodo has been in use for about 12 years.
Kristi Holmes: We recently participated in the development of the code that powers this software, and there are a number of features that make this a very fair and
Kristi Holmes: fair, as in findable, accessible, interoperable, and reusable platform that can be trusted to hold that content of note deposits into Zenodo receive a digital object identifier. So that's a unique identifier that allows them to be citable and trackable. So you can see metrics and things like that about the items in there.
Kristi Holmes: One other feature about Zenodo that's really nice is that they have this feature called communities. And what you're seeing on the right hand side of your screen is the translational impacts community. So it ends up being kind of a branded space where all of our things are together, and we can monitor submissions, and we can support back and forth dialogue on the metadata for those, so that we make sure that those items are as discoverable
Kristi Holmes: and as transparent as possible, and we're actively encouraged. If you have items that you'd like to deposit. Please let me know, and I'll make sure and connect you with the group all right next slide.
Kristi Holmes: This is just a quick
Kristi Holmes: to show you what those metrics can look like. So this is for another community that we have for the generalist repository ecosystem initiative which is run out of
Kristi Holmes: the office for data science strategy at Nih, and we have the same kind of community for our resources there. So just to show you that you can really see the ways that people are connecting with resources, what's being used and so on. So, anyway, just a little glimpse at some metrics, because this is about evaluation next slide.
Kristi Holmes: Okay? So our plans for the upcoming year we want to, of course, make progress on those working group deliverables. We are interested in presenting openly about this work. So if you're interested in anything that we're doing, or if you'd like to learn more, please don't hesitate to reach out, and we'd be delighted to do whatever we can to support that knowledge exchange.
Kristi Holmes: We want to continue engagement. We have had the opportunity to connect with several Ctsa working groups and have had some really fantastic conversations. We're starting to connect with pods with some of the Ecs. So it's been great impact is on everybody's mind, and we want to make sure everybody has a seat at the table.
Kristi Holmes: We also are looking for speakers for the working group meetings. We want to make sure that we give people a chance to share their work. So if you're interested, please let us know, of course, our impact summit, which you'll be hearing more about as plans firm up and then building our community of practice
Kristi Holmes: next slide.
Kristi Holmes: So if you'd like to learn more if you'd like to get involved. Here are some details for you. You can see the links to the mailbox or the sign up form, and then there's a QR code so that you can go to the sign up form as well.
Kristi Holmes: We are also asking for things. So, as I mentioned, please nominate speakers for upcoming workgroup meetings. We'd love to hear different voices. We'd love to hear about projects that you have underway, and then also let us know if you have any materials to share so
Kristi Holmes: evaluation instruments, reports, data, visualizations, logic models, whatever it is you'd like to share. We'd love to be able to help make those resources citable and discoverable.
Kristi Holmes: And then the last slide is really, I just wanted to highlight. There are a couple of questions that we had we recently presented to the steering committee, and we had a couple of questions for them, and I just thought it would be useful for us to share those with you, so that you can see some of the things that we're thinking about.
Kristi Holmes: So if you have input about any of this, we encourage you to either share it with your colleague from your hub, who is on the working group, or reach out to either Manny or I directly.
Kristi Holmes: You know what are the strategic priorities? We should be thinking about. Should we be working directly with particular groups, or, you know, is there is there a missed connection that we should be aware of
Kristi Holmes: how do you want us to report progress? We are eager to look for ways to effectively communicate all of the work from this great group of people. And then always, as with all of these efforts, always interested in the sustainability conversation.
Kristi Holmes: so with that I will hand over the microphone but welcome any questions in the chat and follow up afterward.
Kristi Holmes: Thank you.
Lauren Fitzharris: Wonderful. Thank you so much, Christy, for that really nice overview of your working group.
Lauren Fitzharris: and I don't see any questions coming into the Q. And A. Box at the moment, and in the interest of time I think that we will move to Wayne's presentation. But if you
Lauren Fitzharris: if anyone would like to send a question. Please feel free to put that in the Q&A box. And, Christy, you can keep an eye on it. Great.
Lauren Fitzharris: Okay. So our next presenter is Dr. Wayne, Mccormick and Wayne. I believe that you are going to be screen sharing, and I will stop sharing for a moment.
Wayne McCormack: Okay, thank you.
Wayne McCormack: All right. Thank you. So much for the opportunity to share this update about our working group. The Ctsa Translational Science competency based assessment working group. We have by last count about 58 members from 29 ctsa institutions around the country. And they represent
Wayne McCormack: T program directors, you K program directors, program administrators, evaluators. Just it's a a nice cross section of our workforce development
Wayne McCormack: community and the Ctsa program. And we've been meeting twice a month on 1st and 3rd Wednesdays, although we're kind of in a lull right now, and I'll I'll explain why in just a few minutes.
Wayne McCormack: So why develop a competency based assessment tool? As as many of you probably know, research training has traditionally been on apprenticeship, especially Phd training. And and you know, a big question is always, you know, things like, when is a postdoc ready to lead their own research group, and, you know, get their 1st real job. When is a student ready to graduate? And sometimes we hear mentors say, Well, I know it when I see it.
Wayne McCormack: But wouldn't it be nice if we had a way to really figure out. Are they really ready, based on how competent they are? And so that leads us to the idea of competency based assessment which is very common in health professions, education, so competencies are the ability to do something successfully or efficiently
Wayne McCormack: so. The the goal is then to identify what are the competencies for translational scientists? What are the the knowledge, skills and attitudes that that they need to lead a successful translational research team and identifying competencies is not in and of itself
Wayne McCormack: good enough. What we need to do is to link a list of competencies to some kind of an assessment tool, and our goal is to is to ensure that that's a formative assessment tool, something that's used during training and can support and can support
Wayne McCormack: good open communication with mentors.
Wayne McCormack: So the the 3 goals of the working group over the past almost 2 years now have been to identify what the key competencies are, and and the domains that they fall into to develop a an assessment tool that supports those mentoring conversations, and then to do the pilot testing to really assess the the mentoring value of this assessment strategy.
Wayne McCormack: We focused on the characteristics of a translational scientist because those 7 characteristics each suggest a set of skills. And of course, they are embedded in the Nofos for our T. 32 and K 12 programs, and we also found opportunities to bring in translational science principles into the competencies. So if these 7 characteristics represent translational science, then
Wayne McCormack: what should that translational scientist be able to do? What are the competencies?
Wayne McCormack: I'll say that we we based this work on some previous work that I've done on developing
Wayne McCormack: a framework for competency-based assessment for the training of Phd. Scientists, which was published in 2018, and this paper seems to be having quite some impact based on some Altmetric scoring. What we did previously was we identified 44 competencies just for science. Phd. Training. This was done by a group of folks in the Aamc. Grade groups. Their biomedical science, Phd. And
Wayne McCormack: and postdoc program directors, and and you can see listed there 10 domains that we defined then. And I'm not going to go through this in any detail, and you'll see the slides later. And again, it's important to note that having a list of competencies is not enough.
Wayne McCormack: And so we did propose a framework that we're using in our working group as well to use a milestone approach instead of ratings. And I'll describe that in a few minutes. So the 1st phase of the project was to develop a list of competency, domains and competencies.
Wayne McCormack: And so this work began in the fall of 2023, and went through the spring of 2024. The 7 characteristics of a translational scientist represent 7 of the 10 domains in the assessment tool 5. So 2 of those process innovator and systems thinker are fairly unique
Wayne McCormack: to translational science. But the other 5, actually overlapped with competency domains in the original tool that my colleagues and I developed, and then we also adapted 3 other domains from that original tool
Wayne McCormack: to make sure that ours was really kind of, you know, cutting across all the some of the other skills that we didn't feel were represented. So ethical researcher, resilient scientist and research leader, some additional skills that we wanted to include
Wayne McCormack: the the last part of phase one was, we did a national survey, asking the questions, do we have the list of competencies? Right? Have we missed anything important, and we use that feedback to then finalize the the list of competencies. So we, we asked people if we should include the competency as written, revise it or omit it because it doesn't really fit. And we also asked if if there are any competencies missing.
Wayne McCormack: and we had nearly 200 responses, so most of the hubs were represented, and it was a good mix of trainees, scholars, program directors, and and folks in other workforce development or or evaluation roles.
Wayne McCormack: So we use that information to modify the list of competencies, and there are 44 of them. It just so happened that there are 44 in 10 domains again, that was not intentional. And then here are the 5 that overlap. And then here are the 3 that we adapted from the original science Phd competencies.
Wayne McCormack: So the next phase is really developing the tool, then, so developing an assessment tool based on milestones. So milestones are observable behaviors that reveal a level of competency. And what we did was we used the Dreyfus and Dreyfus model of skill acquisition that they defined 5 levels from novice to expert
Wayne McCormack: and we kind of layered that over the continuum of training that that
Wayne McCormack: our trainees and scholars represent. So going, we defined. And you can define this in different ways, obviously with different people. But we define novices. That beginning Phd student, or recognizing that not everybody's in a degree granting program, maybe a clinician who's beginning research, training with little or no experience. And then moving through, we put the finishing Phd. Student beginning postdoc in the middle. We want them to be competent.
Wayne McCormack: And then we defined expert as a science professional, you know, could be a faculty member in, you know, starting their their lab or research team leader in a in A, you know, in a drug company, for example, or biotech
Wayne McCormack: and the the idea then and is.
Wayne McCormack: we're thinking very specifically about our trainees and scholars, and where they fit. So our Pre Doc T. Trainees would be in those kind of 1st 3 levels, our postdoc trainees somewhere in the middle and Rk scholars off in the right hand side of that that scale.
Wayne McCormack: So if we, if you define these 5 levels of expertise or skill acquisition. Then the idea is to write a milestone, write a brief description of an observable behavior that represents that level of competency. So that was, that's the approach that we took.
Wayne McCormack: and that just shows one example of some milestones. And as you read from left to right, hopefully, you'll see that the level of complexity of those skills is increasing as you move from left to right, and in many cases they build on each other as you move from left to right.
Wayne McCormack: So phase 2, we began writing the milestones actually, while that 1st survey was going. And so we drafted the milestones, and then we did a second National Survey last fall to determine whether the milestones were calibrated correctly. So what do I mean by that? So our, if you look at the average level of self-assessment that trainees and scholars did.
Wayne McCormack: Are the predocs kind of, as I mentioned a minute ago, off to the left and the postdocs somewhere in the middle and the case scholars off to the right. We had 133 usable responses.
Wayne McCormack: and many of the competencies revealed the kinds of trends of self-assessment that we were hoping for where predocs on average were lower than postdocs, and those were lower than K's, and in many cases the difference between the pre-doc t and the K was statistically significant. But for some of them we did not see that pattern.
Wayne McCormack: So if we didn't see that trend or any significant differences at all. Then we took a closer look at the milestone wording, and we tried really hard to write them in a way that did represent that continuum, although you know, we do admit that not everything, you know builds in that kind of a linear fashion. And that's okay.
Wayne McCormack: So that work has been ongoing then, throughout the spring of this year, revising all of the milestones, and it was just completed
Wayne McCormack: in April.
Wayne McCormack: And so what we are doing now is we are gearing up to do a real world pilot test. We're working with the Ccas survey team to put everything into Redcap, and we'll be surveying. Well, I think it's on the next slide. The goal is to complete this pilot test collecting of all the data by the end of the calendar year.
Wayne McCormack: and then we'll we'll move on to distribute it. So phase 3 is this pilot testing. We'll be inviting trainees and scholars to do the self assessment and identify at least one mentor to do an assessment of their competencies independently, we'll combine the information from the self assessment and the mentors assessment into a report that will be shared back
Wayne McCormack: with the trainees and scholars, and we'll ask them to meet with their mentor and and talk about the results. And this is one example of what it might look like with some color coding, showing.
Wayne McCormack: for example, in the top row, the mentor and the trainee agreed on the 1st few milestones, but the mentor actually selected the 4th milestone shown in blue, the other opposite pattern might happen. We might have trainees or scholars who are more confident in their ability than their mentor is.
Wayne McCormack: And so there's, you know, different patterns that you might predict you would see. And the whole point of this is to provide an opportunity for the trainees and scholars to have a conversation with their mentor about their work on on each of these competencies.
Wayne McCormack: so we'll ask them to have that mentoring meeting, and then finally, we'll be inviting them to participate in either a focus group or a very brief survey again to assess the the value of this whole experience for those mentoring conversations.
Wayne McCormack: And so this will spill over into 2026, finishing the data analysis. And we plan to submit a manuscript to Jcts. And we'll be very happy to speak to any of our consortium groups, enterprise committees, other working groups. The administrators have been fantastic, had a wonderful meeting with the administrators at the Acts Conference, and really
Wayne McCormack: look forward to
Wayne McCormack: your help, especially program directors and administrators in getting the word out to your trainees and scholars later this summer, when we're ready to to roll this out, so we probably don't have time for questions. Sorry we're a couple of minutes over, but my email address is there, and I will take a look at the Q. And a. And put in answers that I can in the last part of our
Wayne McCormack: our meeting here. Thank you.
Lauren Fitzharris: Great. Thank you, Wayne. And yes, please take a look at the Q. And A. Box. Further questions, and
Lauren Fitzharris: we will move on to our final presentation by Dr. Daniel Armstrong on the Engaging individuals with disabilities in Research Working group.
Lauren Fitzharris: Danny.
Daniel Armstrong: Thank you. Thank you, Lauren, thank you. And and I really appreciate the 2 working group presentations today really exciting work that you're doing. Thank you for sharing that with us the engaging individuals with disability in research, working group is in its final final 9 months of work. Let's go ahead and go ahead to the next slide, please.
Daniel Armstrong: Which is just disclosure, and I don't have any disclosures here. So next slide.
Daniel Armstrong: our goal of this working group was to inform the Ctsna network about the ongoing working group. And this was really focused on engaging individuals with lived disability experience. So that includes individuals who have a disability, family members who have a disability, close friends and caregivers who work with someone
Daniel Armstrong: with a disability. What we hope to be able to do is share with you the anticipated components of the toolkit that we're developing and understand potential for generalization of our model across other disease and condition research programs next slide
Daniel Armstrong: our overarching purposes provide the context and strategies for investigators to actively engage with the disability lived experience, community in the conduct of research. One of the things we've learned is that while there is a desire and an interest by many investigators to include individuals with disability
Daniel Armstrong: is part of their research group in more than just an advisory committee capacity. There's not a lot of knowledge or skill or experience in how to do that. So we're focusing on the outcomes that both contribute to scientific knowledge and that are meaningful to the research community. What we really hope to be able to do here is provide
Daniel Armstrong: knowledge, resources, and tools and provide a process that enhances the quality of scientific outcomes. And importantly, the meaningfulness of the research to the community that is the focus of the research
Daniel Armstrong: that's being conducted. And so that's 1 of the main benefits on the scientific level. Next slide, please.
Daniel Armstrong: We are aligned with the national training curriculum and the enhancing the impact of clinical trials, priorities of the Ctsa network next slide.
Daniel Armstrong: And this one, this working group has actually been being done in collaboration with the association of university centers on disability. And this is a national organization made up of institutions rather than individuals
Daniel Armstrong: that is focused on disability. It involves centers that are funded as university centers for excellence in developmental disabilities or the Ucedds. There are 67 of those institutions programs that have leadership, education in neurodevelopmental disabilities. The lend programs there are 60 of those have been funded by Hrsa.
Daniel Armstrong: The Intellectual and developmental disabilities research centers, the Iddrcs funded by Nichd 16 of those. And then we include folks with developmental and behavioral pediatric training programs and the autism intervention, research network on physical health or Arp, which just concluded, it's important to recognize that these institutions have a tremendous impact.
Daniel Armstrong: They they provided technical assistance, continuing education or direct clinical services to more than 2 million people last year. And they trained more than 4,300 trainees in 47 different disciplines in disability, education, but not necessarily in research. And so it's a big big collaborative network for us to to be able to work with as we move forward next slide, please.
Daniel Armstrong: It is important that there to recognize that there is alignment. At least.
Daniel Armstrong: institutions have multiple programs. That doesn't mean they have always talked to each other. But we have 9 universities that have Ctsas and a ucedd and a lend, and an Iddrc, 18 have a ctsa, and 2 of those 3 programs and 4 have a ctsa with one of the 3. So we we have a nice overlap of opportunity for our Ctsa
Daniel Armstrong: programs and the Aucd networks to be able to do collaboration at the institutional level next slide, please.
Daniel Armstrong: This is just a a slide of our initial founding work core members. I think we have over 40
Daniel Armstrong: folks now from the Ctsa Network and Aucd, who are working with us on these projects and includes representatives from our hubs. The lend programs, the Ucedds, the Iddrcs, the Arps and we include have included and are adding more individuals with disabilities and family members to that process.
Daniel Armstrong: Next slide.
Daniel Armstrong: This is our deliverable, our primary deliverable was a toolkit that would provide practical and meaningful strategies that would include and fully engage individuals with disabilities and research
Daniel Armstrong: and our guideline, our toolkit will involve guidelines, information materials, recorded webinars and successful examples that facilitate this engagement and includes work or toolkit components in these in these 8 or that will address these 8 areas, development of study questions, study hypotheses, selection of appropriate research designs.
Daniel Armstrong: selection of measures that are scientifically important and meaningful
Daniel Armstrong: participation informed consent, process, data, collection and data analysis and interpretation and outcome dissemination, including authorship of publications. And each the toolkit will include the appropriate guideline webinar web-based resources for each of these 8 elements. And you can see this really is a model for what we do in much of research already. Next slide, please.
Daniel Armstrong: Our second deliverable is at least one publication that addresses the process and context of research participation, including lessons learned about challenges and benefits in a recommended curriculum for investigators. Next slide, please.
Daniel Armstrong: This was our plan, and we are now into the active work of creating these toolkit components and and they have modified. But, as you can see.
Daniel Armstrong: we have one toolkit focused on developing aims and hypotheses, one working on adapting measures and interventions that can be accessed by individuals with disability lived experience. So we've looked at disabilities related to hearing, to communication, to visual component, to motor, gross motor and fine motor
Daniel Armstrong: as well as adaptive language in all areas. So folks are working in that area. We've got a toolkit that's focused on study designs that are practical and that work for people with disabilities and ensure relevance. We need a leader. Our leader with this one was unable to continue. But we will have a section on ethical considerations specific to disability.
Daniel Armstrong: One on understanding context and impact of data, analysis and interpretation. One on recruiting and sustaining participation in research.
Daniel Armstrong: one on writing and publishing together an interesting one on structuring the research environment for both bench and clinical trials, research for individuals with lived disability, experience, and ultimately, as as things sort out in all of the changes that have occurred since January, the how to's of funding
Daniel Armstrong: an individual with disability, experience on grants and administrative supplement possibilities and the like. So a practical how to for the the group next slide, please.
Daniel Armstrong: We've also put together a series of webinars. Our initial one was on May 9, th which was adding meaningfulness to research.
Daniel Armstrong: This one was really to inform and inspire. And all of these are going to be archived on our website for access if you happen to miss it that time. The second one was just a couple of weeks ago. Next slide. This was with mark, slice and heather risser
Daniel Armstrong: and that was focused on building investigator capacity to engage individuals with disability in the research teaching investigators how to do this essentially. And it was a. It was a wonderful presentation that that covered a lot of things that we we normally in our research world wouldn't think about and so again that one will be archive, archived. And then
Daniel Armstrong: as was mentioned as Carrie mentioned a few minutes ago, we have the 3rd coming up on Friday at 2 and this one is going to focus on
Daniel Armstrong: really, it is primarily focused on for folks with lived experience who want to be part of research teams. And it will be designed specifically on how a person with a disability can you know, how can they participate. What do they need to know? How do they learn that real word examples? It'll be an interactive session. So if you're interested, please register for that webinar, I think you'll find it
Daniel Armstrong: be really beneficial. And again, that will be archived next slide, please.
Daniel Armstrong: We included in there a dissemination for a white paper. We have a writing committee. And and actually, the the. I think this white paper is going to be reflective of the process
Daniel Armstrong: that we've gone through for the last year and a half, trying to sort out what the issues are, where the barriers, where the potential solutions were. How do we go through this process, and this group will be working on a paper that we plan to have ready for submission by November. We've also had conversations with both the Journal Pediatrics and the Journal Clinical and Translational Science.
Daniel Armstrong: about a potential
Daniel Armstrong: special issue with at least 10 papers involved. There's strong interest from both of the journals. The current roadblock is funding. I did not include it in our initial work group proposal that we would be doing fundraising, but that's where we're at right now, trying to work on getting some money to help to pay for a special issue
Daniel Armstrong: next slide, please.
Daniel Armstrong: We also realized into the process that we needed to be able to collect some data from both investigators and from individuals with lived disability experience to make sure that our group didn't have a bias or a selective
Daniel Armstrong: experience that limited the kinds of things that we needed to be considered. And so we have developed a 5 question survey that focuses on barriers and benefits of having individuals with live disability experience on research teams, recommendations for additional toolkit components and content and format considerations that that folks have
Daniel Armstrong: for these these toolkit components so that they are accessible and usable. We are asking for multiple responses and an opportunity for narrative editions. And we've asked folks to indicate items that they've personally encountered, and then rank order all the items by importance that has now been worked through with the survey team
Daniel Armstrong: at at Ccas and is has gone through the Irb process and will be releasing that survey throughout our networks in the next few weeks.
Daniel Armstrong: Next slide, please.
Daniel Armstrong: So our working group, timeline, we're completing the 3 webinars that we've planned the survey will be going out in July.
Daniel Armstrong: Our working group is preparing our white paper, the toolkit working teams. We're actually now recruiting individuals with lived experience to be part of these toolkit working groups. And they are back. There are moving along really quite well, and we expect to have the toolkit components ready for final review and dissemination in November.
Daniel Armstrong: Our working group is, is also considering an extension proposal for this as a working group 2.0. So we've got some folks working on that. We are working on negotiating, funding and agreement for special issues. And we were. We were very thankful that the steering committee extended us until
Daniel Armstrong: the end of December. To complete this work, we had a little bit of disruption. This last spring, trying to to figure out what what could be included under the executive orders. But we've gotten that all worked through and and moving forward. So that's where we are. And you know, I have a in. When I was a Boy Scout years ago I had an Indian name called Alopecian. It was fast talker, and I hope I've succeeded with that
Daniel Armstrong: today. So let me see if there are any questions.
Lauren Fitzharris: Well, we are right at about time.
Lauren Fitzharris: to adjourn the webinar. So thank you, Danny. There is one comment. I can send that to you in the Q. And a. But thank you, everyone for these really great presentations, and for the great comments.
Lauren Fitzharris: And our next webinar will be July 23.rd Thank you to those who stayed on with us a few extra minutes. But this concludes our June webinar. Thank you again to the presenters, and we will see you next month.
Lauren Fitzharris: Thanks, everyone. Take care.