Technology Tool Shed

Reflections: What have I learned and what's next

Qualitative research for someone who has spent the last 23+ years focused on reporting things empirically and through quantitative research has been challenging. My pre-educator career back ground was commercial steel construction and there was only "X" is what you do because otherwise "Y" happens. I then went into teaching in the Career and Technology Education (CTE) fields of manufacturing, architecture, and STEM. Of those only architecture has freedom to research and to ask why or explore the feelings. At the beginning if this course that is what to me that's what qualitative was. However, as we have progressed I have come to realize it's more of an observational method. It allows me to observe or ask others to explain to me what something is/means/etc... It then helps take out your own personal bias towards a subject. Qualitative leads to a deeper understanding towards understanding why something is a certain way. Quantitative is still important and needed when it comes to applying the knowledge gained from qualitative in experimental methods. qualitative studies can result in new questions or ideas that you never would have come up with on your own.

I have already begun to start to create a questionnaire and come up with interview questions that can actually be used in my dissertation. For now I think my study will be over Simulation Based Training (SBT), safety, and learning in Career and Technology Education (CTE) classroom and/or industry training programs. After this class I now realize that this may change based upon my findings from observations and/or interviews. As such I have scaled this back from researching from the point of view that it will improve and instead I am asking for other teachers input about SBT:

1) When you hear about SBT be using in education what is your response or feelings about that?
2) Do you think SBT could be used in this education/class setting? For example the one you felt had safety issues or that could be potentially dangerous?
3) What place could SBT have in a learning environment like the one you just told me about that had danger or safety issues?

I then plan on using this data to guide me in asking students similar questions in research of how students perceive it. Then if these two studies support the need, experiment that proves or disproves how SBT affects learning and safety.

What was the experience of doing the shared coding like?

Watching how you processed or worked through the coding process was very beneficial. It helps to see how others do this. The color-coding actually makes a lot of sense, so that everyone working on coding for comparison can see how they ranked or coded that part of the interview. Also deciding as a group what looking for in the interview, so that you can code for example. Student paying attention 1/-1, teacher using technology 2/-2, or students working on projects 3/-3. I realize it’s just going to take a lot of practice and have to do it in a team to ensure rigor or accuracy.

What did you think of the interviewing example with qualitative using semi-structured interview questions? Would you prefer people do more yes/no type questions to speed things up? What did you like about it?

I feel like I am unlearning everything I learned in undergrad and masters when it comes to data. I am actually learning to adore the semi-structured interview method, mainly because it brings to light things that I might not have thought about. It’s great especially in the formative stages to make sure that you are actually focusing on the right things. I would rather spend an extra 3-4 hours on the front end of a study or research topic finding out what matters. Versus wasting 3-4 months working on it, only to find out there was no need for the study or what I was researching had no value to others. What I work on, I do so to help others and not just get published or for my own self. This method can help ensure I am doing this. The person I did my practice interview with, to be honest I didn’t expect him to have any input on Simulation Based Training to help improve safety in classroom/training experiences. However, he was available, willing to be interviewed, and I know he has high technical background. So I knew I would at least learn how he viewed SBT and/or it’s uses in general for education fields. To my surprise he had an experience and great example of where SBT would have been useful for safety. I never thought of it reference to outdoor sports, only as it applies to industrial settings and skills needed. This brought to my attention that maybe I should interview people from all of the different curriculum areas and not just CTE. They may have some thoughts, examples, problems I can research that I otherwise would have missed just asking CTE teachers or industry folks. Quantitative type questions are faster when it comes to deploying, gathering, and “crunching” the data. After all it’s a one or two, A or B, etc… That’s easier to process and prove connections or patterns for study than interview. Interview requires coding and asking someone else to the same to ensure accuracy in what you have interpreted or gathered from the process. It still has a place in my studies but I find this to be equivalent to using standardized tests to judge if a student has learned something. Something that is a “pet peeve” as an educator. That one tests affects students worth for the whole year and doesn’t take into account any of the other projects they have done, ability to apply or use the knowledge. It’s just 1 or 2 days, judging them for the whole year. If this upsets me so much, why would I do the same to my studies and/or participants. I see the need to use qualitative even more so now and to not just rely on quantitative data. They are both needed and have use in conducting a full study. Sometimes I do just need to report age, area of expertise, pass/fail rates, etc… Then there are times I need to dig deeper and find out what could have caused or why it’s important, needed, how it could be used, etc…

How do you know something you saw or read was true? Validity of the research?

When I think of how I know something is true or valid, I first think of to what extent I trust the source it is coming from. There is a saying that "opinions are like ____ and everyone has one." When doing research we have to distinguish between opinions and knowledge, depending on what it is that you are studying that is. This means that I need to carefully select or write the question, ask the question in the most logical manner (survey, interview, etc..), and selection of the data source. I would not want myself to be used as a data source for someone who is studying nutrition and how to best lose weight in women over 40. Simply because I have not been successful at that, I can answer my struggles but not on how to do it. Likewise, asking people about class room management, who have never taught in a classroom before. You have to get to know the people involved in the research, so that you will know if they actually have knowledge of how to do what you are studying. Now if you are studying how it has affected or influenced them versus actually working, then yes you would ask anyone. Your questions and direction of your research is going to drive how you check and/or do your best to ensure validity of your work.

My research area of interest is simulation-based training(SBT) and how it can be used to help teach/train in concepts that are dangerous and/or not financially responsible to offer to those new to it. So that we can ultimately increase safety and production in the real world work force. When I ask what his or her feelings or thoughts are about the potential use of SBT, then this is open to anyone who has taught or trained. However, if I want to find out how well SBT actually works to teach, I need to work with those who have successfully taught or completed the SBT learning. If they have never incorporated or used SBT, they cannot tell me how it worked, did it work, would they change anything, etc… To make sure I am doing my due diligence and practicing rigor, I need to ask people who have experience in SBT at varying grade levels and/or subjects. Obviously, there could be skewed data if I just asked computer science majors versus elementary social science. I have to be aware of where my data could be skewed or where I need to increase or ensure rigor has been practiced, BEFORE someone else points it out to me. Otherwise, why would they continue to read my research or believe any of my findings?

For me rigor comes from knowing up front any strengths or weaknesses that your study could have and finding balance between both. Truthfulness comes from asking people who have experience and not just an opinion about the question. Validity comes from using the best method to seek the data or questions. None of this can be completed affectively in a bubble or on your own, peer review and assistance is a must. Just like we can catch simple spelling or grammar mistakes for others, others can see where we might be too focused on what we think the research will be versus what it actually needs to be.

Blog #5: Qualitative Topics of Inquiry for Simulation Based Training (SBT)

Topic #1: Can SBT be used to affectively meet the four main areas of need for a experiential learning environment?

• Community Centered

Small classroom

Large real world industry, school, etc…

• Learner Centered

What do they bring to the educational experience, not the teacher

• Knowledge Centered

More the big ideas and not just details or skills

Concentrate learning process, development of knowledge in ways that they can use it, and authentic problem solving or real world problems

• Assessment Centered.

Formative – meaningful feedback that is ongoing through the whole process; part of reflection & internalizing of the knowledge; self-assessment process

Summative – comes at the end or conclusion, the grade, yes/no student is ready to move onto next level

Perhaps an argument as to why the experiential  learning method is required first, but the inquiry based process that this method utilizes usually requires no convincing. People all agree that yes you learn by doing. So the question then becomes can SBT be used to provide that learn by doing? Does it provide for building of community? Is it really centered on what the student brings to the experience and how to best provide them with more knowledge or skills? Does it just focus on made up problem or real world scenario that students can understand? Is the student done when they pass a certain level or test, or does it keep evolving to meet even greater needs?

Potential References

Bransford, J. D., Brown, A. L., Cocking, R. R., National Research Council, National Research Council, & National Research Council. (2001). How people learn: Brain, mind, experience and school. Washington, DC: National Academy Press.

Constructivist learning environments. (n.d.). Retrieved from https://leocontent.acu.edu.au/file/ad4c28f6-329b-41fa-a625-bb4dccb18a26/1/html/ped_1_20.html

Topic #2: Do the teachers/students feel that SBT could train as well or better than traditional methods?

• More comfortable because they know that a mistake doesn’t equal real world injury?

• If they do make a mistake, can they learn from it quickly?

Than waiting for let’s say summative test score?

They can try again and again until they get it right?

• Allow more time practice due to limited:

Equipment, Supplies, Lab, Teacher

• Is there still a need or want for the instructor to be there and guide the process?

• Is there still need for learning expectations to be given before start of training or allow the students to set these goals?

This question can apply to teachers, students, administrators, parents, legislators, etc… Do we feel that SBT can do as good job or perhaps better that traditional classroom methods? Do we feel safer because no real world injury will happen? Is feedback quicker? Can student reflect and learn faster? Does it make up for lack of equipment, supplies, labs, teachers? Do you still want the teacher present? Do you still want the learning expectations clearly outlined? All of these need to be followed with the caveat questions of why they feel this way.

Potential References

Magana, A. J., Brophy, S. P., & Bodner, G. M. (2012). Instructors' Intended Learning Outcomes for Using Computational Simulations as Learning Tools. Journal of Engineering Education, 101(2), 220-243. doi:10.1002/j.2168-9830.2012.tb00049.x

Topic #3: What about SBT excites/concerns teachers/students?

• Does it affect student confidence in what they learned?

• Does it affect student satisfaction in what they learned?

• Does it affect motivation to learn?

• Do they feel that SBT is user friendly or have anxiety about it?

• Do they feel that SBT skills really transfer to real world or work place?

• Worried will lose job or not be hired because used SBT?

This question can apply to teachers, students, administrators, parents, legislators, etc… What excites or concerns them about the use of SBT? Do they have confidence in what they learn via SBT? Are the satisfied? Do they want to learn more? Does SBT cause any anxiety? Will the skills really transfer to the real world and equal employment or in teachers case loss of employment? All of these need to be followed with the caveat questions of why they feel this way.

Potential References

Murray, C., Grant, M. J., Howarth, M. L., & Leigh, J. (2008). The use of simulation as a teaching and learning approach to support practice learning. Nurse Education in Practice, 8(1), 5-8. doi:10.1016/j.nepr.2007.08.001

Blog 4 –Different Theoretical Models

Personally I lean towards Inquiry-based learning process or constructivist learning theory. That we learn by actively constructing the knowledge from our experiences and environment. We learn from doing and expereincing. Each learner brings their own past experiences and culture into the process of constructing or learning the knowledge that you are trying to provide for them. It is not so much how they are taught; so much as, it is prior knowledge and how they processed it so that they could understand and retain that knowledge. Which of course means the each person will be different to varying degrees. Which as an educator makes it essential to assess prior knowledge and make sure that the students have connected or constructed a strong base for knowledge, before they acquire more. Of course then you get into who’s view of the knowledge is best? How do you assess to see if they have the knowledge base they have? How do you meet the different needs and levels your students are at? Thankfully I never taught just level or AP students, my classes where all mixed and so I had to start off assessing where my students were at knowledge or skills wise for every unit we did. I learned quickly that just giving pretest was not going to work at least not every time. That it had to be about the skills the students could demonstrate to me. There are multiple ways to draw a circle using ACADD. If I only assessed asking about the two key method, I would leave out the ability or skills using the ribbon or quick access bar. They all work and for students used to 2 + 2 always = 4 this is often hard to get them to understand. For me it is always about can you explain to me how or why used the process you did? Can you perform a task safely and effectively? Then you prove to me you have constructed a good knowledge base and we can move on to the next level. Students struggled with explaining WHY or HOW, but they ended up able to do more when they did this.

Unfortunately, this often put me counter culture of the standardized testing regimen. Having to justify or give proof of how I was helping to prepare students for the state tests. Also, most software certification exams will only accept one way to complete a task. Which is how the programmers themselves would do it and not how the users actually do it. So as much as I hated it, I would have done my students a disservice if I didn’t help them with overcoming the testing only view. I learned to find questions from practice exams and incorporate them into each unit. So that students had the experience with the knowledge in more than just one way and could construct away to turn that into knowledge they could then retain. I just never assessed them only on how they answered questions but made sure that I also assessed based off what they could do to prove they had the knowledge.

If you had asked me if I would be open to learning or at least giving another teaching style/method a try I would have been hesitant or even said no. I would have said something like that I knew what was the best method for my students. However, now I know each student constructs the knowledge gained in a different ways. That doesn’t make the way right or wrong. As long as it’s safe, efficient, and works then its right for them. I’ve gone from a very structured, there is only one or two ways to doing something, to being open to try different methods. There are some things that are factual like certain chemicals will explode when combined or if you don’t follow the proper steps for lighting a cutting torch it can blow up. So I need to ensure that my students understand WHY this result happens and help just aren’t reciting back to me something they learned short term and then forget because it lacks context to them. Why I feel strongly that simulation based training can help students. By allowing them to safely experience, things that could have negative consequence like harming them. Allowing them to build or construct knowledge about the skill BEFORE we utilize the dangerous equipment.

So yes, I would be open to at least attempting to learn or construct my own knowledge about the method or approach. Then I would be willing to work with and apply what was asked of me to the best of my abilities. This is something I feel can only come from experience and the ability to admit that you do not always know everything. When I was younger I thought I knew all that I need to, as I get older I realize I know actually little. Someone having different views or ideas does not negate or endanger my own personal ones. Instead, it is an opportunity for both of us to learn more.

Blog 3 – Ethics and Technology

“Do no harm” applies to more than just medical fields or a line for white witch to say in a

movie. I also applies heavily to education technology use and research. Technology has great implications to either improve or harm those who use it. There are ethical questions when it comes to the use of technology that many do not think about at first.

It is not about technology for technologies sake:
We have think first is the technology use applicable for not only the subject matter, but also the instructor teaching needs or style. The question must always first be will the use of technology in this case must add to or benefit the students. If the answer is no, then you should not use the technology, even if it is the official approved technology or software. Sometimes the technology is not available equitably to all users. Something education constantly has to work on balancing for all aspects and not just technology.

Honesty:
Unfortunately, in today’s age there are more ways to “hack” a system than there are to catch them. You have to be aware and do your best to ensure that the person you are teaching or collecting data from is the actual or real person they are presenting themselves to be. It is sad but identification is needed for proof be it in online or face-to-face interviews. It is not always about cheating for a higher grade. When monetary, even small amount are involved there can be the potential that someone will repeat the study multiple times to receive the funds.

Protection:
We know that we have to protect our participants and provide either anonymity or confidentiality. The smaller the pool size is of participants, the harder this is to do. It is no longer just about protecting the responses or answers they provide. It is also now about protecting their identities from theft by those who can use what you have collected for study and use it to figure out how to access the participant’s social and/or financial accounts. So unfortunately, we have to take great care to protect to ensure that the tools used to collect the data that we provide themselves are not vulnerable to being corrupted and used to harm the participants. You also have to be prepared for people to sign up to participate with the sole intention of access you the researcher’s social and/or financial accounts. Security is no longer just about keeping data or photos locked up in cabinets. It is about protecting and locking down everything. Just think about how much damage could be done if someone hacked into your phone and accessed your data using it. This is reason enough for us to welcome the use of not only passwords but that and bio-metrics like finger print, eye scan, facial or voice recognition's to be used to protect all of those involved.

Unintended Consequences:
We use the technology to improve or make things better. To provide education that a student might not have access to otherwise. To teach concepts or skills that can be hard to learn or possibly dangerous. The problem lies not in the intent of the teacher or researcher to find ways to use technology; it lies in how others use the technology. My own studies focus on simulation based training to teach basic hands on experience to gain skills that otherwise could cause physical harm to learn. Most agree that this is definitely something that needs to be studied and used. What happens if it is proven valid and then school administration decide to only use it and not add in the advanced actual hands on component needed to learn the skill for the real work place? What happens if students only want to learn via simulation and not real world and then get lost in a simulation that is more fun or fulfills a need they have and stops interacting with the actual world. These are not what I think will happen because of my studies, but these are things that could happen and I need to be prepared for ways to counteract and educate during the use of technology to avoid this happening.

Blog Post #2 - What do you think about qualitative research based on your experience gathering and doing initial analysis together?

Ok now I believe that now I have a better understanding of why or how qualitative research is useful. I guess I always confused it with a method that only worked in social applications. For my brain to process things, I have to use examples and for me the best example is when we worked with NASA in the HUNCH design and prototype program. My advanced engineering design students who worked with astronauts to solve issues they experienced on the ISS and it was my job to be the guide and the “why” person. Always asking them why they picked this handle or why do you think Velcro is best used here?

The HUNCH and NASA/ISS experience:

My training in science and engineering still understands quantitative, and quantitative or repeatable experiments is what I have been trained to do all these years. You apply X to see if Y or Z happens, record what happens and then rerun the experiment multiple times to see if you get the same outcome. It does not 100% guarantee that it will happen that way always but if you can reproduce the same outcome, then you can be confident that your findings are valid and true. For example, I know we can only use certain types of metals and no plastics when designing space flight ready equipment. We know we cannot use plastic because it has caught on fire when used in past experiments. There are simple facts or numbers that we follow because of past scientific experiments or experiences recorded in data.

NASA deals only with quantifiable numbers right?
I didn’t realize it at the time but I was teaching and applying qualitative practices to my CHS HUNCH students, especially in the beginning. A qualitative practice happened when my students would sit down with the astronauts and ask them to tell them what “bugged them” the most while working on the ISS. What piece of equipment or process did they struggle with the most? I had the students both write down and video record this so they could go back if needed, not knowing that was a qualitative procedure. Usually they would get a list together of these issues and go back to school with them. Then they’d decide what problem they wanted to tackle for the year and formed teams of 2-4. Once this happened they would begin to research and find anything they could about it, they had to build a folder with 30+ references and come up with questions to ask the astronauts about. Then we’d usually video conference in with the astronauts on the ISS and the students would ask them the questions, ask to be shown what they were doing and be explained or shown how it was a “pain”. Did the astronaut have any ideas or preferences to materials to be used? Then they’d have to get with NASA engineer to get rules that they’d have to follow to meet the safety guidelines.

The students would then begin to apply and build prototypes for what they thought would solve the problem the ISS astronauts were facing or dealing with. They’d present the first draft that year, go through the whole interview process again, ask questions, get input on design changes, etc.. from astronauts in person. Then go back for second round of prototype design. Sometimes refinement and precision manufacturing needed to happen and sometimes they had to start all over again. All based off of what they observed and learned from the astronauts needs.

Qualitative skills and processes learned from the experience:
This ended up ultimately teaching my students and myself that full research and application when dealing with people not machines is a long process that sometimes can take years. I know I had 4 different teams tackle how to best crush food containers for storage over 3 years. For one experiment table to be manufactured and actually deployed to space it took one school I know 3 years and 8 prototypes before they got it right for flight. This is all because you are dealing with humans and their needs, they will not all be 100% the same. Our job is to find the best solution that meets their needs not what we perceived to be the. I never really connected that this was qualitative research because we always built our prototypes based off material and calculations made on what was needed for it to work properly. However, everything that was done qualitative the only truly quantitative work was in selection of materials approved for space flight and the physics mathematical equations students would have to work out to prove it met the safety requirements. For me this is quite “eye opening” and I am seeing where on how I apply qualitative research daily in life and my PhD studies but just wasn’t self-aware that was what I was doing.

CHS HUNCH Program link

Blog Post #1 - What I think I know about qualitative research:

Qualitative research is for finding trends, opinions, observational research. It's about the qualities that make up what you are researching and it can't always be repeated or measured in an experiment. Coming from a quantitative background in engineering/STEM this has been a research method that I have struggled with. I understand how to take data or numbers and find statistics from that. Whereas, with qualitative your more concerned with how you are going to start investigating something you think could be a research problem. Thankfully being in education for 22 years, I have made many decisions about how to best teach my students based off how I have observed them in actions. What worked or didn't, willing to ask questions and find out what is going on. Versus just going by their grades on traditional work like tests and instead look at the skills they show me they can do. This is probably why I am so drawn to to educational technology applications, because I have seen how they improved my students ability to learn. Now if only I can balance that with my empirical nature that wants concrete data or numbers to use to explain the WHY my students do better when technology is used as a tool in my classes. I am hoping that by the end of this class I will find balance and that qualitative research finally makes sense to me like quantitative does.