Sam Torres I SMX Advanced Special: Was AI-Systeme wirklich von deiner Website verstehen

00:00:00: I'll be honest, i've been kind of staying away from

00:00:02: chat GPT.

00:00:04: Our users like for my brand they're not coming from ChatGPT.

00:00:08: that's our target user.

00:00:10: business users are using cloud polar plexity or co-pilot.

00:00:13: yes traffic from LLNs is still considerably smaller than traditional search but it has also exponentially more conversional.

00:00:22: what content are AI crawlers or agents interacting with?

00:00:26: more often

00:00:27: so the MMS text.

00:00:28: There's so many interpretations of how it should work.

00:00:30: Is is an instruction manual?

00:00:34: I think, as the elevator pitch for your company.

00:00:38: Welcome to a new episode of Marketing Meets Technology.

00:00:42: Here you hear insights on digital marketing strategies and technologies.

00:00:52: Have fun with this episode.

00:00:59: LLM crawlers are actually working and what they can see, because I think that is one of the really big topics out there right now.

00:01:05: We have... On our servers.

00:01:08: we see Chatterpity coming by, we see Gemini coming back in, all these other LLmcrawlers and AI crawlers come by And it's interesting how they work Because they do a little bit different to the classic Googlebot.

00:01:19: Then also going talk about rendering.

00:01:21: why this so important?

00:01:23: what has changed there, because obviously your site needs to be rendered for Google.

00:01:26: To see it and understand that also.

00:01:28: four you know for AI SEO It also needs to Be understood What is on the page?

00:01:32: And this can make or break Your ranking.

00:01:34: so That's we're going talk about an.

00:01:35: I have a really special guest here.

00:01:37: My guest today Is someone who genuinely defies The usual job title.

00:01:41: Sam Torres as a developer data architect and a technical SEO all rolled into one and She used that remix to solve problems that most people don't even Know they exist And she's currently the senior manager of technical SEO at Pipe Drive, probably all know Pipe drive pretty big company.

00:01:58: She also in the past founded a really nice agency called TheGray.company where she helped brands turn messy complex data and problems into strategies to actually move the needle.

00:02:10: If you've been around on some conferences like Brighton SEO SMX or popcorn You're probably seen her on stage.

00:02:21: She's also one of the most generous voices in the woman-in-tech SEO community.

00:02:26: So yeah, who could better explain how rendering works?

00:02:29: How quality work?

00:02:30: so it is a real pleasure to have you here Sam and let us get into these crazy topics.

00:02:36: Thankyou so much for having me.

00:02:37: those were some really kind words.

00:02:39: thank you I'm excited

00:02:41: Me too!

00:02:42: What has been working on in the past?

00:02:45: AI... AI SEO and GEO, I don't know what you call it in the US to be honest.

00:02:49: What is that?

00:02:49: The naming you called the whole AI search thing.

00:02:53: uh i think that's still highly contested.

00:02:56: so like my CMO likes to call it GEO Uh...I prefer AI Search because to me that just what makes them no sense.

00:03:03: um But yeah, I think across my team we've probably used all of them.

00:03:08: So...I think it's a fair game right now and i'm still not seeing one really come out.

00:03:13: Um..i did see an interesting study um Not too long ago And It looks like SEOs.

00:03:21: We have all these different names.

00:03:22: but then if we go to Like uh just digital marketers or more broad careers They're actually calling it AI search.

00:03:30: Yeah so that why?

00:03:31: thats what im trying To call because I feel like Thats What the larger audiences is really coming to you and we need to meet them with air.

00:03:37: Let's hope so, because AI search defines it much better than geo does.

00:03:42: in Germany people are using geo a lot but I prefer AI Search Because It Is More What It Really Is Because It Makes More Sense To Say We're Helping With AI Search Than To Say i'm Helping You With Geo Where Everybody's Like what is geo?

00:04:00: But yeah, but there's been a lot of change.

00:04:03: And I would just like to hear your thoughts on what has happened in the last twelve or eighteen months for you and how we've seen changes moving from SEO to AI search?

00:04:13: Maybe some things that have already noticed.

00:04:17: Yeah!

00:04:20: It's a lot of testing.

00:04:25: All this is very nascent, it's in its infancy so things aren't really set and obviously there are lots black boxes when it comes to these AI or LLM platforms.

00:04:35: So I think testing is what we have to keep doing To try understand them.

00:04:40: as far the more standard playbooks that were making right now I do think it's been kind of fun because we're coming back to, like from a content strategy perspective.

00:04:53: We're coming Back To What is the brand or what are The products?

00:04:57: What's the service actually about?

00:04:59: Because i think really what has happened?

00:05:01: and With search recently you know Google were seeing bunch, but then also with LMS.

00:05:07: We need to get back to talking about what we're an expert on and all of this fluff in extra below that you know write a hundred blogs a month.

00:05:15: You know that tactic?

00:05:16: That was used for so long.

00:05:20: It stopped working And now were actively seeing it hurt brands if you have that much extra fluff An extra off-topic content because its diluting your expertise.

00:05:31: So I would say thats been kind.

00:05:35: the nitty-gritty work is figuring out, you know from these now very large content libraries because these companies have been creating content for ten plus years.

00:05:45: What actually matters?

00:05:47: and then also how do we set up systems and processes so that we can be adding unique POVs or just more thought leadership to our content?

00:05:58: Because that's what people want!

00:06:00: That's what their users want.

00:06:01: they want authentic experiences.

00:06:02: So How Do We Give That To Them?

00:06:05: But also at the same time, marketing teams are getting smaller because of AI tools or just because of the impact from what's going on in the economy.

00:06:14: Which you and I talked about a little bit before we started recording.

00:06:18: so it really right now is very difficult but were trying to figure out how balance this idea scaling ourselves and scaling our output but keeping the authenticity and expertise.

00:06:32: So that's a lot of work I've been doing with automation, pipelines... And basically how do we tenx ourselves?

00:06:40: With these tools to then also enable us have time to be testing so we can understand what those tools are like and how they interpret it.

00:06:49: You know..I'm not a fan at figuring out how to game this system But are there patterns or trends that emerge to continue moving in the right direction.

00:07:01: So maybe tell us a bit about testing you did and some findings that are interesting, to share?

00:07:09: Yeah so first thing is really just trying understand what can LMC which I think we're going talk today because of things like yes traffic from LLNs still considerably smaller than traditional search but it's also exponentially more conversional.

00:07:31: So how do we create more of those experiences?

00:07:33: How do you know whether your showing up in the result, what an LLN actually understands about US and entity?

00:07:46: so a lot.

00:07:52: twenty to fifty questions and see what the LLMs know about me.

00:07:56: And I think that's usually a great place to start.

00:07:58: if you don't have tools from there, it is looking into your log files.

00:08:03: so... What content are AI crawlers or agents interacting with more often?

00:08:10: Are able to see any kind of trends between visibility or citations traffic conversions From LLM Traffic To The Content That They're Actually ingesting, and then also just doing those tests of all right talking in cloud or talking in Gemini?

00:08:29: go pull this page.

00:08:30: Can they actually see it?

00:08:32: Now I will say what fetches is obviously very different than the crawlers that might have... Those are two different sets of technology.

00:08:40: Yeah It's been a lot testing like that Of trying to take these data sets And find out if there anything directional we can lean on Because i do think any hard numbers we just can't trust because they're too much of a black box.

00:08:55: We just need to look at the trends, directionally.

00:08:58: where is it going?

00:08:59: How do these

00:09:00: relate?".

00:09:02: So findings from there are really if your site uses a lot of JavaScript you probably not having as much visibility or LMs aren't able to consume content that you have.

00:09:17: so its almost like If we remember what Google was like pre- Twenty-Eighteen, if you had content in JavaScript.

00:09:23: Google wasn't seeing it.

00:09:25: that I even remembered back in those days being this is JavaScript and have to change it immediately!

00:09:30: We're almost back to that just when it comes to LLMs & Agents And do always like to specify LLM crawlers and agents.

00:09:40: They are two very different use cases but they both new audiences that we kind of have to account for, because there are now so many different surfaces and so many ways users might find you.

00:09:53: So all of this has been interesting going backwards.

00:10:00: when it comes to rendering how is your content actually being presented?

00:10:03: How was it engaged with?

00:10:07: while Google can render just fine But it's also always been a question of, are you making with worth their while?

00:10:14: And you kind have to redo that now for LLM crawlers.

00:10:18: That was really long answer and I hope that helps.

00:10:20: Yeah It helps and we have opened two very interesting topics so far.

00:10:24: one the content discussion is definitely something We should look at.

00:10:28: maybe as next question.

00:10:29: on The first question i would have this Maybe can explain A bit more how these different crawlers work.

00:10:35: So You said web fetch you said agents you said the Google crawler But then JetGBT must have a crawler, and Bing might have a crawler.

00:10:43: And there is also this machine that JetGBt for instance calls with the faceted search... ...and it just gives back citations!

00:10:54: So maybe take us through how these different systems work so we can as SEOs and Geos and AI Search experts understand more what's happening?

00:11:06: So to understand how we can make the pages better and make them easier readable, save resources.

00:11:12: Yeah all good questions.

00:11:13: yeah.

00:11:14: so crawlers with our traditional search engines so Google Bing they use headless browsers in kind of you know Make that whole experience?

00:11:25: They've been able to render JavaScript for google.

00:11:28: it was confirmed back in twenty eighteen while The others have followed suit In their own time.

00:11:37: Now when it comes to LLM crawlers, the majority of them are using just simple HTML fetches.

00:11:44: When they're doing any kind of crawling.

00:11:46: so Any of your scripts?

00:11:48: They're not getting executed.

00:11:49: You know with Google we've been talking about now for four years at this point how there is a an html queue and a rendering queue.

00:11:57: or really We call up the crawl queue And then rendering Q. Those are two separate processes.

00:12:03: Basically, for LLM crawlers that second process doesn't even exist and the reason for that is because it's very expensive to do right?

00:12:10: We see that Google's own costs for continuing to crawl the web those have gone up which is also why I think they don't index as much as they used too Because you got a mega worth worth of resources to them.

00:12:23: But yeah in short lm crawlers they're just doing simple HTML fetches.

00:12:28: So all their seeing is that like initial HTML response, so a way to see that how could you recreate that as a user or as an SEO?

00:12:37: Is actually when use go your URL in the browser Go ahead and view your page source.

00:12:44: That's going be what majority of LLM crawlers are saying whereas Googlebot and Bing bot They say What The User Sees.

00:12:57: Oh, I just the developer like

00:12:59: yeah.

00:13:00: Yes there we go.

00:13:01: Whoo brain fart.

00:13:02: um Totally forgot their name of something i use all the time The inspect tool.

00:13:07: Um exactly

00:13:10: So that's the big difference There.

00:13:13: now.

00:13:13: what's interesting though is That you know Like a say thats kind Of the base of LLM crawlers.

00:13:19: they're Just doing the html fetch or request.

00:13:23: There are times where sometimes LLM crawlers access the rendered version of a page, and that's because they've been pulling it from Google's cache.

00:13:33: Or... That's really the biggest reason I have consistently found.

00:13:39: so its not to say if your content is dependent on JavaScript or Rendered that it completely invisible to LLMs but you're making it whole lot horror for LLMs to actually see it.

00:13:52: So obviously that's pretty problematic and anything that still requires like a click or user engagement, Googlebot generally doesn't see those in LLM crawlers certainly don't.

00:14:05: There are some exceptions through these LLMS right?

00:14:08: Gemini has access to Googlebot so can usually get some of the rendered code.

00:14:18: Yeah, it's what we're seeing is LLM crawlers.

00:14:21: their crawl Technology is just not as sophisticated.

00:14:25: There's Google which makes sense That I do think at some point like open AI anthropic Like one of them has got to buy a crawler At some point?

00:14:34: Like I think we're just gonna see that.

00:14:36: Because they need content They need to keep feeding the model like let's like feeding the beast.

00:14:41: they need to Keep going and so that's Gonna be One Of The one of their next avenues, I think to actually getting that.

00:14:49: But for now basically if you don't see it in the view page source an LLN is probably not dependably seeing either.

00:14:59: Now agents get a little bit more interesting right because agents are operating on behalf of users.

00:15:08: so when it comes to agents they can be scripted do all kinds of things.

00:15:14: So they're a little bit, can this agent see this?

00:15:18: Well it depends.

00:15:19: was the agent written to execute and take the resources to get those renderings?

00:15:27: so I think you know its important.

00:15:30: understand that agents again are... It depends on what their user designed.

00:15:37: But I do also think we need as technical SEOs, ASSEOs.

00:15:41: We need to start thinking about how would agents interact with our site?

00:15:45: So like for PipeDrive were a CRM.

00:15:48: so we're gathering the requirements...we are looking at all right.

00:15:52: if someone wanted to sign up for trial using an agent can this system support that right now?

00:15:59: Now the decision of whether or not we will let users do it is above my pay grade and i'll let somebody else make.

00:16:07: But at least to know, can we support this from a tech standpoint?

00:16:13: I think that's something that naturally falls into the Tech SEO wheelhouse because of all these work we've done with crawlers our entire careers.

00:16:22: For ECOM and how are people able to interact If someone's using chat GPT, Spotify has enabled that for you have the options to turn that on.

00:16:44: So I think there are a lot of audiences like you said and The Crawlers is just kind of fun to see how they say the LM crawlers.

00:16:53: it's almost remember what web technology was ten-fifteen years ago?

00:16:57: That's where And it'll be interesting to see how they continue to evolve.

00:17:03: Or like I say, do they acquire?

00:17:04: Do the technology make their own in-house?

00:17:09: Do they get acquired by somebody else?

00:17:11: you know...

00:17:12: Just build themselves ClaudeCode.

00:17:15: That's right!

00:17:15: Claude code is amazing and Fable has been fun.

00:17:18: so i got it.

00:17:19: Okay obviously makes sense.

00:17:21: So the AI overviews K AI mode on Google and Gemini are allowed to use the resources of Google.

00:17:29: And can you see amazing?

00:17:30: Google bot who we all adore love, and pray too get more crawling budget.

00:17:37: What I have absolutely noticed is less indexing going on but will come with that in a second.

00:17:42: So obviously Chachapiti has deal with Bing But they also have a deal with Google.

00:17:46: so there lot do both.

00:17:48: Now let's talk about other one stairs.

00:17:50: i don't know Who Entropic actually falls what perplexity uses.

00:17:55: And there's probably a bunch more, so they must all be stuck in the middle somewhere where they're not allowed to use great resources and this must turn out for them to be way worse results.

00:18:05: or They must kind of like doing a normal call to Google.

00:18:09: I'm just trying to scrape the result Or something

00:18:11: Like that.

00:18:13: Yes i think That's What we've seen is their.

00:18:15: There's A lot of scraping with google.

00:18:16: thats why We saw some Of The changes.

00:18:18: i Think Why?

00:18:19: that n equals a hundred or what?

00:18:24: Yeah, exactly.

00:18:26: Although I also just saw one of the cases got thrown out for Google and a scraper but still for them to best have my knowledge so... I could be wrong on this.

00:18:44: But The biggest source of content is still the common crawl bot.

00:18:49: So the Common Crawl which?

00:18:52: So if you're looking in your log files, it just looks like CCBot.

00:18:56: But that is actually a third party that crawls the web at large.

00:19:01: Google also uses them.

00:19:03: so there's and think about it Like The Way Back Machine or Archive.

00:19:09: It's kind of this third-party unbiased participant here.

00:19:14: still A lot Of That Is What's Being Used.

00:19:19: Like I say, it's still mostly HTML fetches because rendering the web is just really expensive.

00:19:24: Yeah super expensive

00:19:25: And that's why you know Google's one of the few people who can actually afford to do it.

00:19:30: Obviously Microsoft can as well but and my understanding is The relationship between chat GPT in Bing.

00:19:39: They're getting results from Bing But they're not doing as much of their rendering.

00:19:47: That is my understanding bit.

00:19:49: It's all

00:19:50: no conversations

00:19:51: behind closed door.

00:19:51: So

00:19:52: now, of course I mean I find it fascinating that basically if you look at how LLM search or a research works right now is That basically anything that they would read will be taken as one hundred percent re truth?

00:20:08: and Without Google and without being the indexes which have twenty years of experience with spam content Or false content?

00:20:16: The AI results would be horrible Yeah.

00:20:19: So that's what I'm always saying to people when they come to me and say like, okay how easy is it to rank on a search?

00:20:25: And I said It's pretty easy right if you have done the right measures but You need to be ranking in SEO otherwise you have zero chance.

00:20:35: That is really interesting because i think without The experience of this spam team On Googlebot I would not Be able To do any web searches Because it Would be full Of wrong wrong content, advertising content so on.

00:20:50: And I think that plays a huge role.

00:20:51: and yeah it does really interesting.

00:20:54: but have you ever heard of the mechanism?

00:20:57: That works like this... So basically you have Chatchapiti puts out the fanout queries calls Google several times But then it doesn't call the google index directly.

00:21:07: It gets like a search result where they get citations from most relevant pages and uses them in an instant drag database which then computes into an answer.

00:21:20: And that is why there's so little links in their.

00:21:23: yeah, I've also all be honest i've been kind of staying away from chat to bt.

00:21:28: um So yes i've heard about this but Our users like for my brand they're not coming from chad to bT Like.

00:21:38: that's our target users like business users are using cloud polar plexity or co-pilot.

00:21:44: Co pilot is one of the ones That Is giving very little information and it's hard to kind of see where its consuming because we can't differentiate between a big bot.

00:21:56: So that one has been more fun, probably not surprising.

00:21:59: Microsoft is also close the chest on privacy for co-pilot users.

00:22:07: They've got all these great reports coming out now in being webmaster tools.

00:22:13: but yeah And certainly in the US, chat GPT usage is not what it was a year ago.

00:22:22: Yeah of course I mean their competition's really strong Really strong.

00:22:26: I mean Gemini.

00:22:27: A lot people get Gemini for free In the workspace So its amazing.

00:22:32: and then obviously now Claude.

00:22:35: I mean Wow Claude co-worker just insane.

00:22:39: so i think most businesses will be using Claude And once people have gotten used to it, we'd be very hard to move them away.

00:22:47: We're very hard.

00:22:48: so what we see in our company?

00:22:50: We use all of the LLM So we have unlimited access to Germany because we are using Google and then we have Cheshire PT Because there was one other first and now we have Claude.

00:23:00: and I do See a lot of people just enjoying Claude when getting to use the processes and Using co-work and then installing the skills and doing the connectors.

00:23:10: And I know that, that Chatchapati has the same.

00:23:13: It's called Codex but it's a little bit too late you know?

00:23:16: That is an issue.

00:23:18: Yeah as...I do think your right As these brands are building all this context into whatever platform they've chosen.

00:23:26: You're right its gonna be really hard to move off of that.

00:23:29: Exactly.

00:23:30: Okay very interesting.

00:23:31: Let us talk about the content topic A little more.

00:23:34: So um..you said something Because the cost was so high for Google To Index?

00:23:39: Obviously because A million SEOs are spamming the internet with a million content articles every day that I created by Cheshire PT.

00:23:48: Yeah, I think Google has to be very careful of what they look at and spend their time on because everything is just a remix of something that hasn't been before And i'm really interested in your perspective for how you do it right now.

00:24:01: What's the solution?

00:24:02: Maybe as an professional advertiser or our company To solve this problem is liked by AI and Google?

00:24:14: Yeah, I think there's two facets to that honestly.

00:24:17: And that is the content quality and then technical signals.

00:24:21: so Content Quality... We know it because they're starting publish some of them.

00:24:29: They are looking for unique things.

00:24:30: if its not new information If It Is Not A New Fact Then Its Not Going To Be Worth Their Time Right!

00:24:36: They Obviously Have The Index Of The Internet to go with it already.

00:24:42: Now any kind of like proprietary tools, there are some studies out... There's people much smarter than I am analyzing.

00:24:53: how do they do the content uniqueness?

00:24:55: On the technical side though that is where.

00:25:00: Figure it out.

00:25:01: Okay the content that we are creating because I think there's also times where people Are creating good content and its not getting indexed.

00:25:07: And so why is that happening?

00:25:09: Um, and i think It comes down to you.

00:25:11: Are You giving The right signals To indicate That this Is worth Rendering in resources?

00:25:17: This is worth crawl Resources?

00:25:18: um Because of kind Of what We mentioned earlier.

00:25:23: like some of These sites have gotten So bloated they're so big They've got you know so many different clusters or so Many Different Kind of Like random hallways to turn down and get lost in from a crawling perspective.

00:25:36: When your content is tucked away or it's put into these kind of far pockets, if you're not giving the right indicators that this is worth the time then why would a crawler decide to spend resources?

00:25:51: So the reason I say all about?

00:25:53: because a lot of web right now using JavaScript to varying degrees, right?

00:26:01: There's always frameworks that are a hundred percent JS dependent.

00:26:05: And then there's time.

00:26:05: so it's like I just have jQuery installed for fun little interaction with my accordions.

00:26:10: Right who knows We've got everything in between but we are seeing the rise.

00:26:14: from a web development perspective were saying A lot more of these jobs for based framework pop up over The last really three four or five years.

00:26:31: Now, like I say the resources to render those.

00:26:33: Google can do that.

00:26:35: Are we making it worth their while?

00:26:37: It's things like what is available in that initial HTML?

00:26:41: What's what's available In that server request?

00:26:45: are We sending page titles or we having the right meta directives our Wii R The in-body copy links rate art?

00:26:52: are those actually available Or are we waiting for JavaScript renderer to see them?

00:26:56: so they're just kind of like this.

00:26:59: this priority level of what information is being passed, where I have absolutely seen in projects were once we tweaked that content was getting indexed much more regularly because we're making it easier for the different crawlers to know like oh!

00:27:14: This is what this talks about and why its important.

00:27:17: Whereas if you are hiding some those important priority bits of information like your page title or meta directives Like H-one If You Are Hiding Those Behind JavaScripts It Just Looks blank page.

00:27:29: And so why would anybody bother with that?

00:27:34: So like I said, there's really two things going on.

00:27:37: is that Google is tightening up the requirements for Why it would care about a page from a technical perspective?

00:27:44: and then of course The quality?

00:27:46: they unique perspectives?

00:27:48: um i think That's probably where They're spending A lot Of their work From an algorithm Perspective uh and Like I Said There are Much Smarter People Who can Speak to that way more eloquently than I can.

00:27:59: So, yeah...

00:28:01: Yeah cool my question would have been like how do you think they are detecting AI content and separating it from real human content that is actually written from an expertise standpoint?

00:28:11: Because i think there's something really hard because I don't think a human writes better content Like average humans will not write better content then the average AI That's not possible.

00:28:21: The AI just better at.

00:28:24: But obviously an expert human will write at least a new perspective on the topic.

00:28:29: that might not have an absolute truth, compared to in AI.

00:28:32: That might only right.

00:28:33: summary of the ideas that are available and its model or on the Internet?

00:28:41: And I really wonder how they do that right now because we've seen it Least that creating AI content on at scale would work last year.

00:28:51: I just stopped working two months ago.

00:28:53: Right, it's not working anymore!

00:28:55: Um...I do think sentiment and tone is a large part of at least how users are able to determine AI content versus user-written right?

00:29:06: Because its exactly what you were talking about How AI does really good job.

00:29:10: summarizing They also don't often dissent So they're very agreeable.

00:29:16: Yeah Uh..I DO THINK IT IS AN INTERESTING TOBIC THOUGH because LLMs write the way that they do because of the content that they've consumed.

00:29:25: Which means like their whole use of M dash and it's this, not that or not this at that came from real human writers right?

00:29:36: That was something that actual copywriters were creating.

00:29:40: now Of course those are telltale signs And obviously you know we're talking about scalable content.

00:29:44: thats probably going around and identifying and pulling out all these things But I do think it comes down to, are you putting in authentic experiences like quotes testimonials?

00:29:56: Are able put data points that didn't previously exist.

00:30:04: That's a lot of the work at least we're seeing from content perspective when were testing those things or seeing more success than another round up and thank god they going away.

00:30:19: To be honest, it's like this is the most painful thing.

00:30:22: Is to looking for an answer or something and I have a really good example of this.

00:30:28: so obviously we did want to watch some of the World Cup games that were hosted in the US.

00:30:32: there are some great games playing against people Spain playing against teams right Germany losing against easy team but then you had Obviously each country has their own licensing settings and it was an easy question.

00:30:52: I asked Google, where can i see tonight's game?

00:30:55: And AI overviews wouldn't give me an answer so I had to click on these.

00:30:59: um yeah

00:31:01: what are like big magazine titles and what they would do?

00:31:05: They would describe for at least ten sentences What football is ,what the world cup is who's playing tonight but they would not answer Where I can watch the game.

00:31:16: This Is The Worst User experience in my life.

00:31:20: I have ever experienced and then there was ads everywhere.

00:31:22: So, I'm really happy that this is going away because I cannot see it anymore.

00:31:26: And GDPR in Germany is also a huge pain.

00:31:29: you visit the website You have to click ten boxes away To see the content and always pops up exactly.

00:31:35: so you know content layout shift right.

00:31:37: as technical This Is The worst Thing That has Ever Happened to me As A usability thing.

00:31:43: on a Daily basis i open a web page loads fast I want to click on the product, and then the GDPR pop-up comes up.

00:31:49: And they click on it some of...and can't move

00:31:51: on.".

00:31:53: So that's why actually asked Gemini just you know find me whatever and i'll go there.

00:31:58: do not visit pages anymore because im done with this.

00:32:03: Yeah now as an individual user number times like I definitely have experience where I feel Claude knows my professional in my.

00:32:13: Gemini knows everything else about me, right?

00:32:16: Because I've been using Google for so long.

00:32:17: And yeah...I guess in my mind I'm like, Google already does all the embarrassing things about me.

00:32:21: So i'll just continue to use Gemini.

00:32:23: Um But yeah it's Like- I was shopping uh out shopping yesterday and ... I didn't go to google.. I was just in gemini having a conversation.

00:32:34: Hey they're out of stock with this.

00:32:36: Can you recommend an alternative?

00:32:38: um Cause its just It's easier when we have context.

00:32:42: who are your needs?

00:32:44: Yeah.

00:32:45: That's why that's why it will win, that's Why AI Search Will Win because It's Easier and its actually the laziest solutions will always win.

00:32:56: There is this has Always Been in The World sorry to say This but thats why Americans Will Jump In The Car And The Driveway And Drive Five Hundred Meters.

00:33:07: What Is That?

00:33:08: Half A Mile Or Less To The Next Store?

00:33:11: jump out and buy food because it is the lazy option in Germany.

00:33:15: You're not allowed to do that.

00:33:17: basically, you get frowned upon but if it was socially acceptable people would do this in Germany as well.

00:33:23: sometimes they do right?

00:33:25: It's human nature to preserve resources.

00:33:29: doesn't matter mental or on their body side And how we humans survive.

00:33:34: That why will always be an option.

00:33:36: And obviously people are saying like, okay critical thinking will go away and everything.

00:33:39: Yes I agree it might go away but It's just easier!

00:33:44: We're all to blame because we broke the web.

00:33:47: Yeah also i'll be honest on the critical thinking won't go away...it is already

00:33:52: not

00:33:53: as good?

00:33:53: Like..I mean- I have a son he's in school Um ..and im not trying to knock on him But I've friends who were college professors The students.

00:34:03: now Like this is the first time in history we're seeing a decline.

00:34:07: In overall testing scores and IQ, like over the entirety of human history We've seen every generation get smarter.

00:34:16: This was the first times were not seeing that.

00:34:19: And again Not trying to knock on younger people.

00:34:22: Life has been weird for all us right?

00:34:25: But I do think That's already problem.

00:34:31: It's just now we're noticing it more.

00:34:34: because like I mean, I think about my son Constantly is like can't.

00:34:38: I don't let him normally watch YouTube so he'll come to me and say hey Can I find a video?

00:34:44: To teach me this chord on my guitar cool And then you literally have guitar books.

00:34:51: So its things like that of...I Think was the problem already existed.

00:34:58: And I do think, you know we have to tackle this larger problem of how are going to encourage and promote critical thinking.

00:35:07: But also what's gonna happen is there will be a widening gap unfortunately where your're gong.

00:35:16: the people who go above or beyond that can do all those things.

00:35:27: And again, I'm not trying to knock on my son.

00:35:30: He's very smart and very proud of him but you know i think that it's already been going that way.

00:35:38: because is the internet and social?

00:35:42: It isn't just LLMs.

00:35:43: You know there's also a part in me That's really panicking like are we gonna end up In The World Wall-E?

00:35:49: Or in The Eocracity?

00:35:51: Do you see that movie?

00:35:53: Yes!

00:35:56: Yeah,

00:35:58: that's a fair point.

00:35:59: Planets

00:35:59: crave electrolytes?

00:36:00: Exactly!

00:36:02: That is so funny.

00:36:05: I think this has been going on since the newspaper and also TV and then sitcoms and everything Talking about critical thinking...I think there one great place where critical thinking is needed And it inspired and that is the SMX advanced.

00:36:20: Just talking about because i think its fits in red perfectly.

00:36:25: You're speaking at SMX Advanced in Berlin, right?

00:36:27: I am.

00:36:28: So you are coming all the way from Atlanta to Berlin and then talk about these amazing topics which is just amazing by the way!

00:36:37: And um...you've been through so many conferences around the world.

00:36:40: tell us a bit about your perspective on why this time of age it makes sense to go to conference?

00:36:48: Uh..I think its because how much time we spend online getting those inhuman experiences.

00:36:56: Really, I'm super fortunate.

00:36:59: I get to travel a lot.

00:36:59: I do a lot of podcasts and am heavily involved in the women and tech SEO community.

00:37:05: like you said on the SEO Community founded by Noah Lerner You know A lot of great conversations happen there But for sure The best conversations i have are always in person at conferences Because they're so much that is lost from being digital.

00:37:22: obviously we can Do a lot things.

00:37:23: it's very cool.

00:37:25: I would have never been able to maintain friends who live in Turkey without all this technology.

00:37:30: But, um...I think the brain-storming and strategy breakthroughs that really happen at conferences is the best part.

00:37:41: plus SMX always includes a Q&A.

00:37:44: so it also means if you want me with questions or come what's actually troubling maybe you can get some help from speakers.

00:37:53: So, yeah.

00:37:54: I think just in person...I'm also an extrovert so i love people.

00:37:57: Oh

00:37:57: that's amazing!

00:37:59: Yeah- I've been at SMX Advanced a few times and my always said SMX Munich by the way.

00:38:03: but SMX advanced for me is something really special place because all those slots are basically or other talks were in English.

00:38:09: And um it really tries to push the boundaries.

00:38:12: so All of topics they don't start off with what's an H one heading?

00:38:17: They started right You know, okay let's go deep into schema.

00:38:21: That's my first slide.

00:38:22: Okay we're deep in the woods already.

00:38:24: so and that what I really like about it.

00:38:26: So its not Like.

00:38:27: The topics are amazing.

00:38:28: Yeah the topics are Amazing And just experts And i think It is one of the rare places where you can Without any advertising Go Deep Into Topics And then meet real people To be honest Real stars Of the industry.

00:38:39: Really This Is Like If your really In this space And SEO And SCA There's real stars there, like celebrities.

00:38:49: I'd consider you one as well being- Oh that's

00:38:51: really sweet but not the pic.

00:38:53: Yeah!

00:38:54: But do know what i mean right?

00:38:55: So oh yeah...I've

00:38:57: met some of the best people at conferences.

00:39:01: um getting to meet my king in person is always amazing.

00:39:04: Brittany Muller yes um yeah uh I know Viola Eva gonna be at SMX Advanced.

00:39:10: she is incredible and really every time I hear her speak, I'm walking away with just like my head as exploded.

00:39:18: So many ideas so many things i want to try um...so definitely don't miss her talk but yeah this is gonna be really good

00:39:25: Exactly.

00:39:27: So everybody who's listening, you should definitely consider coming to SMX Advanced in Berlin.

00:39:31: it is later this year so definitely check it out.

00:39:34: Sam will be one of the speakers and I can just highly recommend this.

00:39:37: And yeah One more topic i have written down and find really interesting.

00:39:41: You said that at beginning as well.

00:39:44: We're now on a place where we are not trying create mass content.

00:39:48: This strategy was always not great but worked very well especially for SaaS companies.

00:39:52: They would all have same blocks.

00:39:54: They'd all have pipe drive versus sugar CRM and stuff like that.

00:40:01: But the question now is, you said something really interesting.

00:40:04: We're in a phase where we need to create experiences for users And separate experiences from machines.

00:40:10: Maybe let's dive into this topic.

00:40:12: You explain more about your thoughts on how it works.

00:40:20: Yeah!

00:40:21: This goes back some of testing.

00:40:23: Shame at me not bringing up.

00:40:25: I think it is, you know we've got the experiences for users and that's still going to be like your traditional search engines as well where You know.

00:40:33: Is this content engaging?

00:40:36: Are using any kind of CRO or UX principles to make sure That you're grabbing attention?

00:40:42: But there are different things now coming out.

00:40:44: so We have the LLMs.txt.

00:40:47: There.

00:40:48: a lot debate on whether its worth it Or not.

00:40:52: Honestly i'm in place.

00:40:54: It takes like ten minutes.

00:40:56: Just try it!

00:40:58: Also, there was that study that came out from Ramp not too long ago which is kind of when I changed my mind...I used to be pretty anti-LLMs.txt and now just try it because the pages were written in markdown on things better performance for influencing AI citations and mentions than even structured data, which I love structured data.

00:41:29: So that would be kind of the next thing of creating that markdown experience.

00:41:32: because again especially if you have a site that depends on JavaScript for a lot of your rendering—that's really easy way!

00:41:40: A pretty low lift to get your content two LLM crawlers at without having to basically refactor your entire website.

00:41:53: Especially if you're on Cloudflare, they have a beta.

00:41:55: it literally just hit a button and a couple buttons after that.

00:41:58: but its like is this the toggle?

00:42:00: So I think it's interesting to see how the markdown is coming through because Markdown also then becomes a lot easier for agents.

00:42:08: right?

00:42:09: do you have instructions correct?

00:42:12: yeah so Do You Have The Instructions in Your LMS.TXT For An Agent to know how to interact with your site.

00:42:20: Crystal Carter from Wix, she's got a ton of content on this.

00:42:23: it's really good.

00:42:24: I highly recommend.

00:42:25: but just giving those instructions to agents so they

00:42:37: there's so many interpretations of it, how should work?

00:42:39: Is is an instruction manual.

00:42:41: It basically have MCP or is more sitemap that this basically explaining.

00:42:45: you can find my about as page here and you could find my products here.

00:42:48: And this what we do in and always I don't

00:42:51: know all love the above.

00:42:52: i think i think it is a You Know ,I Think of.

00:42:55: it has its elevator pitch for your company.

00:42:59: So if an LLM is going to say like who was pipe drive?

00:43:03: Or What does pipe Drive ?

00:43:05: Like, what's the thirty seconds?

00:43:07: I would tell them and then where to like... Where do I want them go?

00:43:10: To get more information.

00:43:12: For example we just launched a WhatsApp integration that is absolutely in my LLMs.txt because i wanna make sure thats getting picked up ummm.. In the LLMs.

00:43:21: So I think it's- It'a little bit of that.

00:43:23: And Then its also instruction manual for here how to interact with site.

00:43:28: Here's How to Do Different Things Which Is Gonna Be The Agent Instructions And Also Dovetails Into MCP.

00:43:34: They're very closely related in this perspective.

00:43:38: So yeah, I think it's a combination of all them.

00:43:44: now for us like to say we don't have A whole lot of instructions for agents yet because that still business decision That being made above my pay grade.

00:43:56: so For us is mostly just here who are what you specialize And then like, here's what we know are the most common questions that users have.

00:44:05: So that's it put in ours.

00:44:06: but I say Crystal Carter has really good content on how to write agent instructions for LLNs.txt.

00:44:16: so yeah to answer your question

00:44:17: Amazing!

00:44:18: I love it okay?

00:44:19: I didn't know discussions were gonna go to LLM TXT.

00:44:22: But very interesting because my perspective is changed as well.

00:44:25: One year ago i sat actually at a conference and wouldn't use Because nobody was recognizing it.

00:44:31: Now I would say actually the same thing, like if its easy for you to implement You should definitely use that.

00:44:36: but obviously If It's hard To Use then you Should not use That right?

00:44:39: And it really depends on how complicated is to integrate.

00:44:44: But as you just said Its more Like an elevator pitch so you can Just quickly ride it and Then its done.

00:44:50: So in Germany we have this concept of grounding pages which Is very interesting where We basically talk about how we build about us page that has the entities description.

00:45:01: And it's very similar to what you just said with the LMS text is?

00:45:05: Yes, yeah I think those are worth it and like I say its worth testing this all kind of unfurling before.

00:45:14: none of us have crystal balls.

00:45:15: We don't know exactly whats gonna happen.

00:45:18: um i think also the requirement for making sure your structured data sound It has been raised.

00:45:27: Like, it continues to be an important part because its just really easy-to consume data.

00:45:33: and the easier we can make for these crawlers or bots understand our site and who they are... The better it's gonna go!

00:45:43: I always try.

00:45:44: think about developers made bots And the ones trying to do their work.

00:45:51: And as a former developer, I feel like i'm allowed to say we are inherently lazy people Like you were talking about before.

00:45:58: There is reason that I automate As much of my life and it's because I do want to sit down more often.

00:46:05: You know, if these bots we have to assume that they are also inherently lazy.

00:46:11: That They just want to get the answers as quickly and as efficiently as possible.

00:46:15: So the more We make that easier The better.

00:46:19: And then with your structured data it is.

00:46:20: you Know do you have that full entity mapping right?

00:46:23: There's a lot more in schema work than Just what Google accepts.

00:46:26: I think people should really investigate that and see like Are there Better definitions for who we are What we Do?

00:46:33: It may not mean A SERP feature But it probably means increased understanding about who you are and what you offer.

00:46:41: And then that also dovetails to a new standard, that's come out called NL Web.

00:46:47: It is actually founded or made by the same guy who MadeSkima.org.

00:46:54: he's doing with Microsoft.

00:46:56: an NL web is just another format for instructing how interact with your website in a natural language format.

00:47:07: It's very similar to schema, it is pretty easy for developers to implement and so you know the majority of time its just like here s the github repo tell your engineering to do this!

00:47:22: Is usually instructions.

00:47:23: as technical SEO I wouldn t expect attack.

00:47:25: SEO actually let code put that But that is a new standard.

00:47:30: That's coming out and because it does have the backing of Microsoft, And It Is from this one gentleman.

00:47:36: I cannot pronounce his name correctly so i'm not even gonna try to butcher but you know he's obviously changed The world A couple times over with schema.org and all work He has done for web standards.

00:47:50: So yeah there are lot Of these things that Are Coming Out testing to see how it helps, we'll see what wins out.

00:48:01: And I will circle back to the LNs.txt argument...I was right there with you!

00:48:06: Why would we do that?

00:48:08: This is silly.

00:48:11: but also robots.tXT as a standard only finalized in last five years.

00:48:18: So, and how long were we using robots.txt before that happened?

00:48:22: A lot longer than five years.

00:48:24: so you know this whole argument of nobody's using it.

00:48:27: It's not standardized.

00:48:28: a Lot of the internet isn't standardized.

00:48:30: That's why we have like can iuse.com?

00:48:33: because each browser renders things differently.

00:48:36: um So why not?

00:48:38: why not test it in trial?

00:48:40: And what he said is pretty easy to write up.

00:48:42: I actually used cloud to write ours.

00:48:44: Yeah No, but it's that.

00:48:48: That what?

00:48:49: So you've already obviously written in your life so many redirects right.

00:48:54: three or one redirect like in a HD access file.

00:49:00: You make one mistake and the whole file is broken And It gives no reporting.

00:49:04: where this?

00:49:06: Yes

00:49:06: I have spent hours testing complicated hdx files with going line by line.

00:49:14: This has changed a lot.

00:49:16: Now I just push that into Claude and it's like we're done.

00:49:20: And, uh... It is easy.

00:49:22: semantic mistakes can be found so easily but also logic mistake.

00:49:26: That something genius!

00:49:33: I want these files to be locked out, make a great regular expression in the robots.txt file and summarize

00:49:42: it.".

00:49:42: Oh yeah?

00:49:43: And i think that is great because they mistakes are seen on robots txt files as insane and on hdx files this should reduce by a lot.

00:49:52: oh yeah i uh regex i feel like i don't even write myself anymore which i guess its kind of sad.

00:49:59: um

00:49:59: cause i

00:50:00: used to love them.

00:50:01: but

00:50:01: Yeah It was a good

00:50:02: skill.

00:50:02: But yeah..i dont need too...and also Oh, man.

00:50:07: Coding and especially JavaScript which I could probably give a whole nother hour long podcast episode on why i hate javascript.

00:50:18: but like so many times coding that myself you just miss semicolon Yes And the whole thing breaks.

00:50:27: The error code is NOT helpful at all.

00:50:33: It really has changed the game So much Yeah.

00:50:37: That's why, yeah I really like that.

00:50:38: so schema.org if i have a biased opinion on this?

00:50:41: It think it is very hard to implement for most sites and they don't say right now some tags help but others are kind of like.

00:50:54: I'm not sure whether or not the data supports them.

00:51:00: as

00:51:03: soon as time will tell It's also a very easy rabbit hole to fall down.

00:51:09: never come out of because there are so many different types.

00:51:11: Oh, yeah So I do think that's one of those things where you know the The intermediate SEOs versus the seniors.

00:51:18: I think really A lot of times the difference between those is just then knowing when to win to crawl out at the rabbit hole.

00:51:25: What have you gone far enough?

00:51:28: But You Know definitely I've seen a lot of times where just adding this little bit of definition really clarifies who you are and what you offer, especially when it comes to anything that's your money or life.

00:51:42: So finance healthcare being as clear and concise but precise as possible can help so much in making sure.

00:52:00: And like I said, I am biased.

00:52:01: I love schema and think it's great.

00:52:03: um i do have a lot of thoughts for how to implement It make it more scalable.

00:52:07: Um yeah maybe that will be the nest The next smx.

00:52:10: so we'll talk about That.

00:52:11: Yeah

00:52:12: um did We miss anything now?

00:52:14: experience?

00:52:14: We Have To Create For Their Machines.

00:52:15: Did We Miss Any Points There?

00:52:18: um Let's see ellens at txt markdown Schema.

00:52:24: No no I Think That's it.

00:52:26: I Do think it is Interesting Though That We're Going Back To This idea of having two separate experiences.

00:52:32: And there is a part of me that really doesn't like, um because I can remember when pigeon came out and it was just like hallelujah.

00:52:40: We no longer have to have pages dedicated to misspellings.

00:52:43: Um

00:52:44: That's great.

00:52:46: But you know i think at the end of day we also need to accept that Users are now going to be interacting with our websites through their chats Yeah which is not going To Be The Same Thing as actually going to our site.

00:53:00: I think Mike King even has a prediction that brands won't even need websites

00:53:07: in the future,

00:53:09: which is a bit mean.

00:53:12: This will happen

00:53:13: and it'll take half of time for yellow pages disappeared but... It's gonna take some time!

00:53:20: To be honest if you look at large corporations every large corporation listening right now can definitely relate or feel offended.

00:53:29: But they are so big that they probably have just launched their page maybe two years ago.

00:53:35: They're like, oh yeah we had a web page then re-launched it and now finally the version we actually wanted And I'm not saying this is going to be Apple or Microsoft there way far ahead but i am talking about these really big corporates like BP.

00:53:59: Go there with a new technology.

00:54:00: It's going to take less time than I did the first time, right?

00:54:03: So internet took them twenty years and i will not take him twenty years.

00:54:06: it would probably only Take five but i do Not see.

00:54:10: so There Will be some kind of version where you as A company have To host information about yourself And You Have to host Information About your products Your offering The things You stand for the opening Times you have and Some other stuff.

00:54:27: As long as there is nothing like a business Facebook, where you just upload your data and you're done.

00:54:34: This will kind of be the server Where your website has hosted on And it can't Be Just an mcp.

00:54:40: It Can Be The Business Entry In Google But Most Probably At The Mound Will Be The Website.

00:54:46: So I Understand Mike Hundred percent.

00:54:48: He's so smart.

00:54:50: There Is Truth To This But It Will Take Some Time.

00:54:55: Also i Feel Like What You Just Said.

00:54:56: Maybe it'll be the next product that meta comes out with because I feel like they're just pumping out all kinds of products.

00:55:02: Some of them, I'm like, that was an interesting choice not what i would have done but go you?

00:55:06: um.

00:55:07: so yeah I agree.

00:55:10: So if she think about blogs dying?

00:55:12: All these on the blocks?

00:55:14: there was a real blogger community out there.

00:55:18: It's gone!

00:55:21: Medium, it went into Instagram and went to Facebook.

00:55:24: It went in to TikTok.

00:55:26: that's where the bloggers are now.

00:55:27: That's what the content creators.

00:55:29: so bloggers were content creators old content creators right?

00:55:33: So they were assembled on these platforms And this platform made a huge money out of this.

00:55:38: but obviously like all these unique experiences have vanished and now there aren't walled gardens.

00:55:43: I think one other products could be exactly this Like hosting your data for AIs maybe on Google or maybe on on chat about you who knows And that's how you get presented, right?

00:55:53: Somehow maybe a vetting process or something.

00:55:55: I do see this as possible future.

00:55:57: and then you don't need a website.

00:55:59: if your content creator... You just have Instagram and three million followers who only needed a link tree!

00:56:10: It is going to be interesting because we're talking about LLM crawlers.

00:56:17: they are gonna need to start rendering JavaScript so that they can get more content.

00:56:22: But then from there, how do they also start consuming video content?

00:56:26: Because it's exactly like you said... That where most creation is happening.

00:56:31: Google obviously can lean on a lot of the data that they get from YouTube.

00:56:35: So I think Google here for ever

00:56:37: They're in such great position.

00:56:39: Oh!

00:56:39: There really are.

00:56:43: And obviously they consume Video Content.

00:56:45: because if you upload a video on Youtube and make captions But it'll be interesting to see like, how does that continue to grow?

00:56:56: How do technology growth become more affordable.

00:57:00: How is consuming start impacting answers?

00:57:04: and I mean yeah... SEO's never been boring!

00:57:10: And thats

00:57:10: one of my favorite at least favorite things about the job.

00:57:16: No actually its one of favorites but sometimes makes a little tiring.

00:57:20: Yeah, it does.

00:57:22: Sam It's been amazing talking to you.

00:57:25: really thank You for all these answers and insights And I'm really looking forward To seeing you at SMX advanced.

00:57:33: Everybody who is listening should definitely connect you.

00:57:35: Sam where can we connect?

00:57:37: If somebody wants a get in touch

00:57:39: You can find me on LinkedIn and then also in those two communities, Women In Tech SEO.

00:57:44: If you identify as female or non-binary please join.

00:57:48: it's free.

00:57:48: we're amazing super supportive.

00:57:50: We do all kinds of fun stuff.

00:57:52: And there is the SEO community by Noah Lerner.

00:57:54: I'm pretty active with both so that you could always ask my questions.

00:58:01: It's been a real pleasure.

00:58:02: And yeah, let's see where it takes us.

00:58:04: in one year we'll have more answers and probably much more questions.

00:58:07: but that so many

00:58:08: more questions.

00:58:09: yes

00:58:12: Thank you so much for taking your time For this smx special and Yeah Have A Great Day!

00:58:16: That's Probably the start of Your day now In Atlanta and for me is nearly The end.

00:58:21: So take care and See You soon.

00:58:29: thank you so Much.

00:58:32: Bye bye See you next time!