Coding can be a complicated process but only when it comes to setting it up because you have to ensure quality. Getting a group of three people together and agreeing upon the coding to be used is imperative if you’re going to end up with data that is accurate and useful. I know this as a “norming session”, and this is something we do in my department at least once a semester. We use the rubric that has been established to be used to assess online course design. The whole department of Instructional Designers each reviews courses that have been pre-selected, we make notes and “rank” according to the rubric. Then we sit down as a whole and go through the process of explaining what we gave and explaining why. I have found this is essential for our group so that the twelve of us can clarify and then come to an understanding of the most balanced critique or analysis of what is best practice in course design as established by the department. There will always be some critiques that are higher or lower than what the group norm ends up being but that is normal and helps make sure that the process is still authentic but balanced. This is important practice because when an ID sits down with a professor to go over the results, we need to listening to what the professor is explaining as to why they have tings set as they do. Sometimes it because they just don’t know how to setup or use a tool and need our help doing this. Other times it can be because what they are doing makes more sense to them and their students. There is no one perfect way to do something, it all depends on the experience and the needs those in that class. As an ID I must make sure I am helping my professors meet the needs they have, not as just defined in a rubric.
I see this process be essential for any team of three that is coding. This participation of three coders helps meet the need to triangulate the data for the most balanced understanding or accurate data coding. I see the best method to establish norm is to perhaps separately code the first 3 pages of transcript or 5 minutes of interview, then come together to discuss what and why you coded as you did. This incorporates the more postmodern methods of making sure that data collected based on the participants knowledge and not off the knowledge or “expertise” of the researcher that can contaminate the data. That will help ensure a “norm” for the group and if someone is overly high or low, you can explain reasons why. If it is not justified or the explanation not accepted and agreed upon with the group, then you know that you need to access how you are coding. This epistemic practice or approach to data collection allows those researching or coding to take the time necessary to relate the knowledge or data gathered and helps ensure they understand and are able to construct accurate results or knowledge gained from this. This is better than just taking the average score of the group because it helps you to ensure rigor in the coded data. Data is no good, not matter how much time you spend collecting it if it’s isn’t quality. If you don’t work to ensure quality then you will end up providing fodder, so to speak, to those who still view qualitative research as being controversial and less reliable than quantitative. Quantity does not trump quality, especially in research. We have to make sure that we are doing everything we can to ensure quality especially in how we code and report the data we have discovered and not what “we want to find” based upon our own potential bias towards a subject.
I have discovered that this process of ensuring quality is a process that is not quick, if you want to do it properly. I have recently scraped data form Twitter over simulation-based training in an effort to find what the current attitude is towards it. I started with 787 data points and once I took the time to remove those that were most likely to have been generated by a bot I ended up with less than 200 Tweets or text data to analyze. Nowadays it is easy for someone to write a program or bot that auto-populates information based upon their own agenda or product trying to “sell” to others. As researchers we have to be aware of this and take steps to ensure the data isn’t that is a result of this isn’t used in our analysis because that is supporting the bias of a small group versus studying the truth of the larger whole population. To do this I removed Tweets that came from users not curated to at least 20 lists because when you are on a list, this means your more likely to be an actual person with actual information that others find to be of worth. I also removed and text that ended up being a ReTweet and using the most original one to then qualitatively code the attitude as I saw it. Now I need to find two others who would be willing to also code, to ensure reliability and quality of this data.
Post #5 : Quantitative- Positivist versus Qualitative- Non Positivist.
I had never before heard the term “wholest” (hope I spelled that correctly) but I had positivist, when used to describe how one approaches research. It makes sense though because in education there has finally been a shift more recently into studying and meeting the needs of the “whole” child. That the snap shot of the standardized test, which to me is positivist research probably at it’s worse. To only use, this to gauge a student’s knowledge growth isn’t fair because so much can change in a year. The test does not take into account or know if significant changes have happened to that the student in the past year or heck even weeks. Things that can’t be measure from looking at the child. Perhaps the students’ parents just divorced, broke up with significant other, has to now work 20+ hrs to help family met bills, etc… Can you say “ah-ha moment”?!? It just now clicked for me just why qualitative analysis is important. We need to see the whole or big picture. We need to study more than just the surface of what we can see and really try to dig into the “WHY” of it. When I first started teaching I had a “W-H-Y” poster up in my room that was for W-hat we are learning or doing; H-ow is it useful to you and how we will go about doing it, and Y-es you will this will be for a grade either soon or use it later on in class or life to do “XYZ”. I had made this up and used it when explaining my lessons to students. Because I realized that explaining why we were learning something, other than it is a TEKS that Texas said I had to teach. That I needed to explain better to my students why I was trying to teach them this and not just give it as an assignment because it was required for my gradebook. That they needed a point of context or reference so that they could start to internalize and really learn.
Qualitative research is the WHY and I need learn to embrace and utilize it because it leads to deeper understanding beyond the surface level of what I can see or easily compare in a pre/post test. I need to use it so that I can really learn about the subject and not be biased off my experiences or senses. It would be easier to not have had this “ah-ha moment” as to WHY I need to use this type of research. It is because positivist quantitative research is easier to conduct and explain than qualitative research. ARGH
Class tonight itself hasn't changed the way I view qualitative research, probably because I just had Dr. Warren for LTEC 6512 Qualitative Analysis this past spring 2018. I still think more quantitatively because of all my years in STEM/engineering field. Thankfully I have also been in education for 23 years and the qualitative makes sense to me because the understanding of WHY something is the way it is or WHY you do"things this way" is more important than knowing facts. I need to teach students how to think and when it comes to my won research that is the role qualitative serves or acts is for me. It forces me to dig deeper than just numbers and their relations, but as to why they are related. This has led to me focusing my research to the attitudes that people have towards SBT in education/training.
The project you had us do on creating a subjectivity statement where we had to answer the following really helped me start to form and understand my own potential biases and how to find ways to counteract or at least recognize them and to try to not fall for my own pre conceived notions. Those were the:
1) clearly define what studying or the What is
2) What does this mean for my reflection activities and design?
3) What else do I expect to find or see?
4) What do I already know about this topic?
5) Challenge of what I think I already know?
Ideally to really avoid bias I need to have 2-3 others code the same data I collect, preferably with someone who thinks that SBT does not have application use in education/training. If we get the same results then I know my data is good. Hopefully in this class I will find a classmate who might want to team up with me on coding a) Twitter data I have already scraped and b)forming & conducting a survey/interviews about this subject.
Post #3: CMDA and qualitative data analysis potential use in my dissertation
As I have advanced in my PhD studies one potential free resource for data gathering has been Twitter. I use Twitter for professional uses and as such fall more under the "stalking side", in that I follow people who I have greater knowledge in certain subjects. People who post about things that I want to learn more about or that I feel I can use for educational purposes. Plus the whole being able to search for something via its hashtag, or pound sign for those of us who are older, is a research gold mine. This form of micro-blogging where the number of letters or words you can use constricts the Twitter authors Tweet into what really matters the most. If it is retweeted or liked by others then you know it has struck a cord is of importance to them in some way or another. You can't easily quantify or measure what exactly that importance is and that is where the qualitative analysis needs to be utilized. The extent to which Twitter is now used in things like politics, marketing, social presences is something that intrigues me and I want to see if I can use it to discover if their is belief or feeling if you will about the use of simulation based training in education, CTE specifically. What way do they think SBT and education can be used? What impact do they think it will have, be it positive or negative? What does the public think are pros or cons of using SBT in educational settings? Are there differences in how vendors, teachers, students, parents feelings towards the use of SBT in educational settings?
In my nativity I thought gathering Tweets by subject or hashtag would be easy. I quickly learned otherwise. By the way there are many companies who all they do is data mine Twitter and get paid quite well to do this. I can understand why as getting the coding to run in the online tutorials I found for Python, Tweepy, Twurl, Ruby, R studio just never worked for me. I can see why a company would pay others to mine the data and provide them with an analysis report. It's not as easy as you and I thought. Since I am a student and educator, paying for this just wasn't in my budget and goes against my "I can do this" attitude. So I spent 20+ hours researching ways to just get data, not even beginning to study that data. I finally found in my searching https://birdiq.net/twitter-search/, which allows you to pull up search topic and then export it to Excel for further cleaning up and analysis into programs like SPSS, after you code the data. All the other ones I found would only go back 7-10 days, I wanted to get at least a years worth of these in hopes that I can find if there are any trends on how SBT is being viewed.
I am currently in the process of coding data according to the search terms, if it was made by vendor, educator, parent, or student. Was it retweeted or a retweet. If it's a RT go back to the original tweet and find out how often it was RT. Then I can analyze the actual tweets to see what qualitative data I can find, which I know will actually take the most time and I need to find a fellow PhD student who is interested in this same data, so that we can both study it and compare our findings to ensure reliability. Hopefully in this class I will find that person.
PS: If you are looking for just some data visualization tools about what is being Tweeted recently, you can use these tools. They don't export but they will allow copy/paste or print to PDF for use.
https://www.exporttweet.com
Post #2: What I learned about CMDA from the assigned readings & class discussion.I guess that I already knew that most online interactions take place by using text. After all nowadays people text more than they call. Social media like Facebook, Instagram, Snapchat, etc...often use a photo and then text explanation or message. So that part didn't really surprise me and makes sense to me the we need to have study approach or research style such as Computer Mediated Discourse Analysis because this is how we can access or ensure that communication happens that leads to knowledge gained or relationship being built. What stood out the most to me was the reminder that "text can only tell us what people do", that we shouldn't use it as a way to measure who that person fells or thinks. Unless of course they specifically say I fell XYZ because of 123 (Herring, 2004).
Also for me "grounding' or how the online group finds a common ground that allows them to effectively communicate with each other. In face to face environments we can use face and body language that lets us know how it is going (Paulus, 2007). Online we don't have that unless it's a video conference, perhaps this is why we had the birth of emojis? Just an point to ponder.
Interesting to me is that to have a good CMDA research question it shares the same four main characteristics as a good qualitative one:
1- Empirically measurable for data: That the text evidence provides examples of when this happens. The example is you can measure how often jokes where told but you can't measure if they were having fun.
2- Isn't trivial: It shouldn't be something that is already known or proven as fact and it should be beneficial or something that others also would want to learn more about.
3- Motivated by hypothesis: The question shouldn't be worded so that the answer is already given. Often the researchers has a hunch about something and this should be used to drive the hypothesis.
4- Is open ended: These are not questions than can be answered by a simple yes or no. It should be more of the what, why, when, where, who, how style.
Lastly temporal sampling over a time period, preferably at least a year, will result in some observations that can be made about these interactions made by the same people/group or person over a period of time. Allowing to observe if changes happen or not.
What this has done for me is helped me to realize that I need to study the frequency that simulation based training is mentioned on Twitter by how often tagged in education, STEM, and then CTE. Track if it is from an individual or company marketing product, individual are better for me to study because they aren't trying to sell a product. Then finally if it's well received by others. If it gets retweeted or liked by others I can use that to measure that it had a positive reaction with people on Twitter.
Herring, Susan. (2004). Computer-mediated discourse analysis: an approach to researching online communities. Designing for Virtual Communities in the Service of Learning. 316-338.
10.1017/CBO9780511805080.016.
Paulus, T. M. (2007). Online but off-topic: negotiating common ground in small learning groups. Instructional
Science, 37(3), 227-245. doi:10.1007/s11251-007-9042-5
Post #1: What is my personal view of research?
a. What is my general worldview in terms of what I think can be known, why that is the case, and how we can best understand the world?
The statement "the more I learn, the less I know" is so very accurate. Just because I think I know the answer or correct way, doesn't mean that it's the only or even the best way. I've learned to listen and build relationships first, so that I understand where the other persons perspective because when I do this, I always learn something new. I know that I need to be open to learning and new experiences, if I am ever going to be able to help others.
b. What is research to me? What is its purpose? Do I prefer numbers or narratives or both?
Research is a way to gain and then share knowledge. I may have an idea or what I think the outcome will be, research is needed to prove or disprove this. Honestly the majority of my career numbers have been what I prefer because they are easy to understand, they don't change no matter your language, culture, etc.. However, after 23 years in education I have learned that I need to know more than just the number. I need to understand the background, the why, the how so that the numbers have a deeper meaning.
c. What is your main focus in terms of what you are planning for future research? Are you higher ed, K-12, corporate, or other? Are you looking to switch your focus to a new setting?
To be honest I'm still not 100% sure what my focus is going to be on. It depends on where my career takes me in these next few months. I'm finding that I more invested in the doing or practitioner part of education than I am in research. The thought of constantly researching and writing doesn't equal as much joy as when I help a someone transform into a teacher or conquer a new skill that they couldn't do before. I know that we don't have enough teachers and our students suffer because of this. Why I'm drawn so to simulation based training/AR/VR so much. I see it as being a tool that can bring more chances for learning to others who otherwise wouldn't have the opportunity to learn it. I also see it as a need because of the safety issues that are faced in many lab based courses and in industry. I taught wood and metal "shop" to grades 7-12 for seven years. As a teacher I wish I had such a way to allow my students to practice a skill without worrying that if they make a mistake they cut off something or blow up something. I then moved to HS engineering and graphic design for twelve years where I had students struggle with learning concepts that when I could get them access to a 3D model of it = they got it. These last four years I have been in higher education as an instructional designer for online courses. Where I have seen the need. For me I only know that I strongly believe simulation based training has the potential to help so many learn new things.
