63 stories
·
0 followers

Using a Google Sheet to Build Search Queries for Unknown Topics

1 Share

I spent a lot of time building tools at work. There were things we needed to automate, but I didn’t have a server or any kind of programming platform to put things on. Because of that I discovered you could do a lot just using Google Sheets.

I built a tool that let you scan in ISBNs and get the title and author of a book. I built another that let you put in the title of a movie and get a plot summary and list of actors. I designed another one that let you scan in a CD and get recommended genres and sub-genres. (That one I had to hire someone to program as the implementation was a bit beyond me.) There’s no telling how much time those tools saved me at work, and they left me with a slight mania for making useful spreadsheets, though it’s not really one of my job functions anymore.

Last week I got an e-mail from a reader named Joe asking for help researching a topic he didn’t know a lot about. Then at about the same time, I read about an add-on that would let you import Wikipedia data into Google Sheets. And I thought, “Hey, this might be a nifty way to quickly build Google search queries when you don’t have a lot of information.”

And so the Web Search Query Builder 5 Million Google Sheet was born. (The “5 Million” part is in honor of the Hydraulic Press Channel, which seems to name all its original tools the dup-de-dup 5 Million. Visit YouTube to see the Rock Maker 6 Million (it’s an upgrade) turn hair into powder.)

You can try the sheet at https://docs.google.com/spreadsheets/d/1hl0Ku9eqLIcr3piiz5DI_bbeZnQhZP-4eMLRBFLWJ44/edit?usp=sharing . I encourage you to make a copy of it for yourself and play with it. I’ve found it’s useful for getting a quick overview when I run into an unknown concept or name while doing ResearchBuzz; it’s much faster than searching for it “cold,” with no contextual hints. (And since I run into topics and names I don’t know constantly, I’m hoping this will save me some considerable time.)

PLEASE NOTE VERY IMPORTANT: If you do make a copy of it and (and as I said before, please feel free) you will need the Wikipedia and Wikidata Tools add-on for Google Sheets. It’s free and you can find it here: https://chrome.google.com/webstore/detail/wikipedia-and-wikidata-to/aiilcelhmpllcgkhhpifagfehbddkdfp?hl=en 

In this article I’ll give you an overview of the sheet and explain how the tabs work.

What the Heck Is This

What this the heck is, is a sheet that uses the Wikipedia and Wikidata Tools add on to draw data from Wikipedia in response to a query and then build from that data a set of Google search queries which are tweaked slightly by domain limitations and thematic keywords. In the case of this particular sheet, I’m including limitations to the .edu and .gov domains, and adding additional keywords with the hope of finding more teacher-friendly resources. The tool is set up in a series of five tabs.

Tab 1 – Queries

screenshot from 2018 05 08 17 48 41

 

The first tab is for entering your search term. General searches like “cow” will get you so many results as to be useless. I find that names and events and somewhat obscure concepts are the most fruitful; these searches for example brought good results:

Great Patriotic War
Bangladesh Liberation War
Mzwakhe Mbuli
EMDR

Underneath the query box you’ll see count results for your query; enter your search term, and these boxes will briefly reset to 1 and then update with the number of queries returned, thesaurus lookup results, outbound link topic results, and category results. There’s a maximum of 500 items per line. If you find that you’re getting 500s or 100+ for everything, your search may be too broad. I do find, though, that popular culture figures tend to get very high numbers no matter what because they’re linked to so many things.

Once you enter a search term, all the other tabs will update.

Tab 2 – Wikipedia Data

This tab displays all the results that are pulled from Wikipedia and put in this sheet. The data here will be used in building search query URLs in the following sheets. I find that glancing at just this sheet can give me a general overview (very general) of a name or concept.

screenshot from 2018 05 08 19 47 26

Tab 3 – Search Engine queries

This tab presents you with a number of Google search URLs. The first set uses the first five lookup keywords to create general Google searches. The second set uses three keywords at a time to create a more specific query. (It works like this: the first URL incorporates the 1st, 2nd, and 3rd row of Wikipedia query results, the second incorporates the 2nd, 3rd, and 4th row, etc.)

The third set uses five keywords at a time. That one can be so specific as to give you no results!

Hold your mouse pointer over a URL and you’ll get a little pop-up box with the option to open the URL (you’ll see what that looks like in the screen shot.) Unfortunately the URLs aren’t really directly clickable; that’s a Google interface decision and not something I can change.

(Also note that if you run a search that has a limited number of results from Wikipedia, all of the search URLs won’t populate with keywords. They’ll still be on the page, they’ll just be empty.)

screenshot from 2018 05 09 05 09 16

Tab 4 – Search Engine TLD-Domain Restricted Links

Tab 4 takes the same kind of searches done in tab 3, but adds domain limiters so the first set queries is restricted to .edu domains only and the second set is restricted to .gov. There are simple and medium queries here, but the “deep dive” queries of five keywords at a time are removed. I found doing searches that large AND restricting by top-level domain tended to produce few to no results.

screenshot from 2018 05 09 05 09 25

Tab 5 – Google Queries for Finding Learning/Reference

This set of Google queries removes the domain restriction but adds a couple of keywords designed to get the results to go in a more scholarly/learning direction. This isn’t perfect and the kind of result you’ll get will depend a lot on your initial search.

screenshot from 2018 05 09 05 09 35

This Could Go a Lot Further

When I initially developed this, I didn’t know how useful I would find it. But I showed it to my friend Kathy Jacobs, who looked at it and very helpfully broke it in a couple of places (when you make things like this, it’s invaluable to have someone technical enough to poke at it, break it, and then explain clearly to you how they broke it) and she liked it a lot. So maybe you will too.

Obviously this could go a lot further. I could add an option for specifying domains (top level or otherwise) that you want to restrict your generated searches to. You could specify your own keywords. Maybe customizable country restrictions for your searches.

Let me know if you find this useful and would like to see some upgrades. I’ve only done a little spreadsheet work since my job changed, and I realize I miss it!

Thanks to my great patrons at Patreon who are supporting ResearchBuzz and affording me time to write these articles.















Read the whole story
smahoney
137 days ago
reply
Share this story
Delete

How to Rands

3 Shares

Hi, welcome to the team. I’m so glad you are here at $COMPANY.

It’s going to take a solid quarter to figure this place out. I understand the importance of first impressions, and I know you want to get a check in the win column, but this is a complex place full of equally complex humans. Take your time, meet everyone, go to every meeting, write things down, and ask all the questions – especially about all those baffling acronyms and emoji.

One of the working relationships we need to define is ours. The following is a user guide for me and how I work. It captures what you can expect out of the average week, how I like to work, my north star principles, and some of my, uh, nuance. My intent is to accelerate our working relationship with this document.1

Our Average Week

We’ll have a 1:1 every week for at least 30 minutes no matter what. This meeting discusses topics of substance, not updates. I’ve created a private Slack channel for the two us of to capture future topics for our 1:1s. When you or I think of a topic, we dump it in that channel.

We’ll have a staff meeting with your peers every week for 60 minutes no matter what. Unlike 1:1s, we have a shared document which captures agenda topics for the entire team. Similar to 1:1s, we aren’t discussing status at this meeting, but issues of substance that affect the whole team.

You can Slack me 24 hours a day. I like responding quickly.

If I am traveling, I will give you notice of said travel in advance. All our meetings still occur albeit with time zone considerations.

I work a bit on the weekends. This is my choice. I do not expect that you are going to work on the weekend. I might Slack you things, but unless the thing says URGENT, it can always wait until work begins for you on Monday.

North Star Principles

Humans first. I believe that happy, informed, and productive humans build fantastic product. I optimize for the humans. Other leaders will maximize the business, the technology, or any other number of important facets. Ideological diversity is key to an effective team. All perspectives are relevant, and we need all these leaders, but my bias is towards building productive humans.

Leadership comes from everywhere. My wife likes to remind me that I hated meetings for the first ten years of my professional career. She’s right. I’ve wasted a lot of time in poorly run meetings by bad managers. As an engineer, I remain skeptical of managers even as a manager. While I believe managers are an essential part of a scaling organization, I don’t believe they have a monopoly on leadership, and I work hard to build other constructs and opportunities in our teams for non-managers to lead.

I see things as systems. I reduce all complex things (including humans) into systems. I think in flowcharts. I take great joy in attempting to understand how these systems and flowcharts all fit together. When I see large or small inefficiencies in systems, I’d like to fix them with your help.

It is important to me that humans are treated fairly. I believe that most humans are trying to to do the right thing, but unconscious bias leads them astray. I work hard to understand and address my biases because I understand their ability to create inequity.

I heavily bias towards action. Long meetings where we are endlessly debating potential directions are often valuable, but I believe starting is the best way to begin learning and make progress. This is not always the correct strategy. This strategy annoys those who like to debate.

I believe in the compounding awesomeness of continually fixing small things. I believe quality assurance is everyone’s responsibility and there are bugs to be fixed everywhere… all the time.

I start with an assumption of positive intent for all involved. This has worked out well for me over my career.

Feedback Protocol

I firmly believe that feedback is at the core of building trust and respect in a team.

At $COMPANY, there is a formal feedback cycle which occurs twice a year. The first time we go through this cycle, I’ll draft a proposed set of goals for you for the next review period. These are not product or technology goals; these are professional growth goals for you. I’ll send you these draft goals as well as upward feedback from your team before we meet so you can review beforehand.

In our face-to-face meeting, we’ll discuss and agree on your goals for the next period, and I’ll ask for feedback on my performance. At our following review, the process differs thusly: I’ll review you against our prior goals, and I’ll introduce new goals (if necessary). Rinse and repeat.

Review periods are not the only time we’ll exchange feedback. This will be a recurring topic in our 1:1s. I am going to ask you for feedback in 1:1s regularly. I am never going to stop doing this no matter how many times you say you have no feedback for me.

Disagreement is feedback and the sooner we learn how to efficiently disagree with each other, the sooner we’ll trust and respect each other more. Ideas don’t get better with agreement.

Meeting Protocol

I go to a lot of meetings. I deliberately run with my calendar publicly visible. If you have a question about a meeting on my calendar, ask me. If a meeting is private or confidential, it’s title and attendees will be hidden from your view. The vast majority of my meetings are neither private nor confidential.

My definition of a meeting includes an agenda and/or intended purpose, the appropriate amount of productive attendees, and a responsible party running the meeting to a schedule. If I am attending a meeting, I’d prefer starting on time. If I am running a meeting, I will start that meeting on time.

If you send me a presentation deck a reasonable amount of time before a meeting, I will read it before the meeting and will have my questions at the ready. If I haven’t read the deck, I will tell you.

If a meeting completes its intended purpose before it’s scheduled to end, let’s give the time back to everyone. If it’s clear the intended goal won’t be achieved in the allotted time, let’s stop the meeting before time is up and determine how to finish the meeting later.

Nuance and Errata

I am an introvert and that means that prolonged exposure to humans is exhausting for me. Weird, huh? Meetings with three of us are perfect, three to eight are ok, and more than eight you will find that I am strangely quiet. Do not confuse my quiet with lack of engagement.

When the 1:1 feels over, and there is remaining time I always have a couple of meaty topics to discuss. This is brainstorming, and the issues are usually front-of-mind hard topics that I am processing. It might feel like we’re shooting the shit, but we’re doing real work.

When I ask you to do something that feels poorly defined you should ask me for both clarification and a call on importance. I might still be brainstorming. These questions can save everyone a lot of time.

Ask assertive versus tell assertive. When you need to ask me to do something, ask me. I respond incredibly well to ask assertiveness (“Rands, can you help with X?”). I respond poorly to being told what to do (“Rands, do X.”) I have been this way since I was a kid and I probably need therapy.

I can be hyperbolic but it’s almost always because I am excited about the topic. I also swear sometimes. Sorry.

If I am on my phone during a meeting for more than 30 seconds, say something. My attention wanders.

Humans stating opinions as facts are a trigger for me.

Humans who gossip are a trigger for me.

I am not writing about you. I’ve been writing a blog for a long time and continue to write. While the topics might spring from recent events, the humans involved in the writing are always made up. I am not writing about you. I write all the time.

This document is a living breathing thing and likely incomplete. I will update it frequently and would appreciate your feedback.


  1. Speculation: there is an idea in this document that you’d like your manager to do. Thesis: Just because I have a practice or a belief doesn’t mean it’s the right practice or belief for your manager. Suggestion: Ask your manager if they think my practice or belief is a good idea and see what happens. 
Read the whole story
smahoney
200 days ago
reply
Share this story
Delete

Thread

5 Comments and 10 Shares
Since the current Twitter threadfall kicked off in early 2016, we can expect it to continue until the mid 2060s when the next Interval begins.
Read the whole story
smahoney
365 days ago
reply
Share this story
Delete
5 public comments
satadru
364 days ago
reply
❤️
New York, NY
infini
365 days ago
reply
he he he
Asia, EU, Africa
bluebec
366 days ago
reply
This is such a niche reference. I am so happy I know what he's talking about.
Melbourne
alt_text_bot
366 days ago
reply
Since the current Twitter threadfall kicked off in early 2016, we can expect it to continue until the mid 2060s when the next Interval begins.
Covarr
366 days ago
reply
I'm still waiting for a film adaptation of the Pern books. A real one, not that Eragon nonsense that merely steals ideas from Pern.
Moses Lake, WA
crc32
366 days ago
Agreed. Though it could be better as a miniseries or full series like GoT. Lots of material to cover...
silberbaer
365 days ago
Eragon is a film adaptation of... Eragon. Which I felt was far more similar to Star Wars than to Pern. Yes, we need Pern movies.
Covarr
365 days ago
Eragon wasn't a terribly faithful adaptation, though. The book already borrowed a few ideas from Pern, but not enough for it to be a problem. The film, for whatever reason, borrowed a lot more.

Regardless of seniority, every good manager will:

1 Comment and 3 Shares
  • Order pizza.
  • Give feedback.
  • Listen.
  • Take out the recycle/trash. Not a metaphor.
  • Need to find additional things to delegate.
  • Try to support their direct reports to eventually become better than them.
  • Consider constructive feedback regardless of who delivers it.
  • Do what they say.
  • Have regular, un-cancelable 1:1s.
  • Protect their team, push for greatness, and prepare for the future.
  • Get their hands dirty when called for.
  • Focus on helping their team to be wildly successful.
  • Put people first.
  • Give a shit.
  • Relentlessly hustle for their team.
  • Tell the truth.
  • Care.
  • Feel deeply and profoundly awful for disappointing someone on their team.
  • Make mistakes and learn from them.
  • Remove fear
  • Be the bullshit umbrella and not the bullshit funnel.
  • Remember (and maybe learn from) the time when they weren’t a manager.
  • Say please and thank you.
  • Work harder than their employees.
  • Educate.
  • Provide consistent and predictable structure.
  • Back up their team when they say “no” to something.
  • Not be a prick.
  • Model the culture and spirit they want to develop in their workplace.
  • Translate corporate bullshit into normal-speak.
  • Empower their staff members.
  • Move thing out of their way, including yourself.
  • Regularly feel self-doubt.
  • Be an advocate.
  • Be an ally.
  • Amplify the good in people.
  • Fight the grapevine confusion.
  • Be first the first to metamorph to the chrysalis phase.
  • Define reality and say thank you.

(Sourced via the fine humans on Twitter.)

#

Read the whole story
smahoney
624 days ago
reply
Share this story
Delete
1 public comment
digdoug
622 days ago
reply
I'm trying, Ringo. I'm trying real hard.
Louisville, KY

The candy diet

1 Share

The bestselling novel of 1961 was Allen Drury's Advise and Consent. Millions of people read this 690-page political novel. In 2016, the big sellers were coloring books.

Fifteen years ago, cable channels like TLC (the "L" stood for Learning), Bravo and the History Channel (the "History" stood for History) promised to add texture and information to the blighted TV landscape. Now these networks run shows about marrying people based on how well they kiss.

And of course, newspapers won Pulitzer prizes for telling us things we didn't want to hear. We've responded by not buying newspapers any more.

The decline of thoughtful media has been discussed for a century. This is not new. What is new: A fundamental shift not just in the profit-seeking gatekeepers, but in the culture as a whole.

"Everything should be made as simple as possible, but no simpler."*

[*Ironically, this isn't what Einstein actually said. It was this, "It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience." Alas, I've been seduced into believing that the shorter one now works better.]

Is it possible we've made things simpler than they ought to be, and established non-curiosity as the new standard?

We are certainly guilty of being active participants in a media landscape that breaks Einstein's simplicity law every day. And having gotten away with it so far, we're now considering removing the law from our memory.

The economics seem to be that the only way to make a living is to reach a lot of people and the only way to reach a lot of people is to race to the bottom, seek out quick clicks, make it easy to swallow, reinforce existing beliefs, keep it short, make it sort of fun, or prurient, or urgent, and most of all, dumb it down.

And that's the true danger of anti-intellectualism. While it's foolish to choose to be stupid, it's cultural suicide to decide that insights, theories and truth don't actually matter. If we don't care to learn more, we won't spend time or resources on knowledge.

We can survive if we eat candy for an entire day, but if we put the greenmarkets out of business along the way, all that's left is candy.

Give your kid a tablet, a game, and some chicken fingers for dinner. It's easier than talking to him.

Read the short articles, the ones with pictures, it's simpler than digging deep.

Clickbait works for a reason. Because people click on it.

The thing about clickbait, though, is that it exists to catch prey, not to inform them. It's bait, after all.

The good news: We don't need many people to demand more from the media before the media responds. The Beverly Hillbillies were a popular show, but that didn't stop Star Trek from having a shot at improving the culture.

The media has always bounced between pandering to make a buck and upping the intellectual ante of what they present. Now that this balance has been ceded to an algorithm, we're on the edge of a breakneck race to the bottom, with no brakes and no break in sight.

Vote with your clicks, with your sponsorship, with your bookstore dollars. Vote with your conversations, with your letters to the editor, by changing the channel...

Even if only a few people use precise words, employ thoughtful reasoning and ask difficult questions, it still forces those around them to catch up. It's easy to imagine a slippery slope down, but there's also the cultural ratchet, a positive function in which people race to learn more and understand more so they can keep up with those around them.

Turn the ratchet. We can lead our way back to curiosity, inquiry and discovery if we (just a few for now) measure the right things and refuse the easy option in favor of insisting on better.

       
Read the whole story
smahoney
626 days ago
reply
Share this story
Delete

Questions to Check Cognitive Biases

1 Share

I'm reading both Kahneman's "Thinking Fast and Slow" and Michael Lewis's "The Undoing Project" at the same time, which is fun.

A friend and I were having coffee today and he made me agree to write down twelve questions I thought might aid in helping us check our cognitive biases outlined in the books.

Of course, I am not a Nobel prize winning psychologist or social scientist and I have done no work to verify whether these questions actually accomplish this at all or where they might even harm our ability to off-set cognitive biases, but I said I'd do it so I did it.

I'm working off of Kahneman's conclusion that there isn't any way we can actually remove the biases and instead must figure out how to work within them. The only advice Kahneman gives in the book is to bring other people into the question and seek their advice, so that's my first one, but I don't think that's the only one.

Anyway, disclaimers now aside, here are twelve questions. I broke them out into two groups: "normal" questions which we can apply to just about any question we might have, and "mathy" questions which tend to focus on statistics (the original topic of Kahneman and Tversky's research was focused on whether humans are good inherent statisticians. The answer? NO!)

Normal Questions

  • Who else can I bring in on this decision to check my answers?
  • If I'm wrong, what would that look like?
  • How can I change the context or reframe this question?
  • What other evidence can I gather before I make a decision?
  • Am I overcompensating to avoid loss?
  • Am I oversimplifying the question?
  • Am I being overconfident in my ability to make this decision?

Mathy questions

  • What is the base rate?
  • Is the sample size statistically significant?
  • Have the results regressed to the mean or is there a lot of variance with each new added result?
  • How different is this answer from random chance?
  • Am I trying to fit this to an oversimplified or inappropriate model?

What do you guys think of them? Where are my own biases coming into play to manipulate these questions? Email me at mike@mikeshea.net to let me know.

Read the whole story
smahoney
629 days ago
reply
Share this story
Delete
Next Page of Stories