Thomas Petit - Data culture for startups (#13)

Opener (00:08):

Welcome to what the DATA podcast with your hosts, Mitch and Leo. Hello

Lior (00:17):

Everybody. And welcome to another episode of what the data podcast. And today we have with us Thomas. Hey Thomas.

Thomas (00:24):

Hi Lior , very happy to be here.

Lior (00:27):

I'm so excited to have you on the podcast. Actually, I think one of the brightest minds in the mobile industry nowadays, tell us a little bit about what are you doing today?

Thomas (00:35):

Thanks for this intro Lior. I'm an independent consultant I work almost only with apps and almost only with non-gaming apps and I pretty much specialized in the subscription business model. So that's still a lot of verticals, interesting verticals, education, health, and fitness, food, and drinks, news photo and video, like lots of interesting verticals. And I have a background in user acquisition, so I still do a lot of sort of helping and consulting on that. It became a little bit broader over time as my interest grew. So I'm also looking at onboarding flows, retention, strips, and paywalls. Alrighty. And lately a lot of attributions just because it's a hot topic at the moment. So yeah, I'd say not narrow narrow array of type of apps and lots of topics. Very broad. I like it.

Lior (01:31):

That sounds awesome. So let me ask you that you meet in quite a lot of companies, right? So you're working with a lot of companies. I know. Where do you see the most issues with data with these companies? Is there any lines you can connect between them?

Thomas (01:45):

I mean, I, I work with a lot of very early stage people and it's true that most of them, not necessarily that they don't have to necessary knowledge, but like more, it takes time to have a solid data infrastructure in place and have the own data warehouse. And maybe tooling tooling is a little bit expensive at the beginning. If you want something like, I don't know, Redshift and Looker, you have to implement it. So for a lot of early stage clients, it's sort of the cost. And the time of implementation means that they basically work on very poor tools and with very limited set of data, basically they have to do Firebase and they're a bit weak, but the problem here, we never solely itself. I mean, it's just a question of time as company grow and become like more siruse about everything. And they realize that that is important. I think the one big issue that I face very often is wrong data and clean data. I contaminated data in the sense that a lot of people want to be the data driven and they say my decisions are based on data, but nobody's questioning the, how the data they use is sourced and remixed and Tinder. And other like basically does not even have to weigh on that. And a lot of people using it, they never questioned its validity. And I think that's the biggest issue. Like another, another issue both when it's at scale and in the early phase. So I'd say it's even more common than because it's for everybody is that viruse function used various sets of data or metrics or even source of truth. And so everybody's working on their own typically. I don't know, like I had a case last week where the acquisitions and say, Hey guys, the caution to trial is dropping. And then we go to the monetization team, like the team in charge of onboarding and payroll and so on. And they say, no, it's growing. And yeah. And then I looked at both things and everybody was going in the wrong direction. They say, but it's just like, I took us a while to realize what it is that but yeah, I'd say the, the everybody looks for what it was.

Lior (04:18):

That's one of the biggest, I think common things that we see also in the field of data, everybody have their own tool to check information. Everybody has their own Excel. Sheet is a lot of fun.

Thomas (04:30):

Yeah. I mean, in some cases, you know, like some tools that are particularly suited for, for specifics, I don't know a PM will prefer to have amplitude, but maybe, I don't know, the UAA team will only look at where the aggregate, the cost, maybe on the MEP side or all their here. What's important is that guys like you and people in charge of data actually can kind of, we have these different tools using similar sources and similar like similar blocks of data so that whatever ways they look at them, at least it will match in trend. They are different, slight difference between tools, but that's a big, big job to align everybody in the company because yeah, I mean, that's a situation that happens very often. And I, and I don't think anyone's to blame here, like, Oh, you're doing it wrong or whatever is different departments have different needs in terms of data. So, well, they, they go for what fits their needs better or sometimes traits the gap was the other department. I think that's a big problem,

Lior (05:38):

You know, that's I heard the conversation on the podcast a couple of days ago again, and we were talking exactly about this problem because PM, as you said, have amplitude, then the marketing adjusts and then the management received information from the finance team, Who were collecting it from the marketing, the product. And they were combining some data that looks fine. This person that you will be on the podcast. And I will reveal it later. [inaudible] Was saying that one thing that they realized was that the more tools you use, the more complex it's becoming too make decisions, then the more the management actually hates you and is walking in the marketing department for quite a while. Now. I'm not sure if the episodes will come before you, after you. So this way, I'm not saying the name yet, but people are going to hear about it. And the funny part about it was that seriously, especially when consultant coming into the company, one of the biggest challenges that he mentioned is bred that he has also consulting a frame that had been done is to understand where the data is coming from and how they're making decisions. And it's a huge mess. Do you feel you have the same thing each time you joining a new company to console them?

Thomas (06:49):

Yeah, almost everybody. When I join a trick to understand, like, what are they who is using what, like to start with, like where the are they actually using that one is looking at amplitude that one, that one looks into firebase, that is looking into their own database. That one is looking at raw data, like, okay. And then I tried to like, something I do very often is I'm not trying to solve that problem. I'm not trying to convince people. They should switch tools. Like this is like, sort of as an external, it's hard. I mean, you can recommend when they want to implement a new thing, but like making them change, it just don't do it. So one thing I learned is like sort of that data QA checks. I mostly run for myself to try to make sure what I'm looking at, which is basically exporting sets of like a few common events that they would have first open, maybe complete the sign up, maybe number of activity, like some very basic events, not too many of them because it takes time. And I extract them like with a, with a couple like breakdowns. Typically I look over time maybe by platform. And I tried to put like three, four, five of them in parallel and say, what I'm looking at here is not, am I seeing a huge discrepancy? Is, is the trend match? I mean, at the end of the day, one to 110, the other a hundred and the other 90. Yeah. That's always going to happen. What I want is that they always move in the same direction in the same proportion, because if they don't and that's a massive problem in one of these that I was doing, I still didn't figure out why, but something that was perfectly correlated forever with still a 10% gap started decorrelating up to the point that one is reporting double as the other right now. And yeah. And a lot of people making decision based on that, which were.

Lior (08:52):

That's horrible. Right. When you realize in such a thing.

Thomas (08:55):

Yeah, man, that's the worst. Because as I say, it's not that people's grew up. I mean, and that's why I like doing more your dad. His problem is you make decisions that are logical. I mean, it happens to me many times to realize later I completely up. But at that moment it was the right decision to make. Like, based on what I knew is just, there's a lot of things that I didn't know that I didn't know.

Lior (09:19):

Let me ask you that. So when you're arriving and you starting doing the test, how much time actually second from your onboarding phase? Is it something that is an hour or two or is it a little bit longer to understand which data is right or wrong? And if the trends are going in the right direction,

Thomas (09:34):

Probably try to limit it to like one hour or two. Basically. I mostly do it on my own will like, it's not something I can necessarily invoice or I dunno, I do it for myself. So I try to keep it simple. And they're just like big checks. Sometimes they come later because you arrived as a hot problem. You start on like, there's no onboarding. Basically this goes straight at it. A lot of people, I work with them either one time or ongoing, almost forever. I have a few clients that almost don't churn at all. I have the same couple of apps that I'm helping for the last two years. And in this case, it's not really at the onboarding is once in a while when I don't know, we're seeing something is, is weird, is different to before then maybe I would run a bit of a health check like this in the middle doesn't have necessarily to be at the beginning. Plus, you know, also system evolve. Sometimes people implement something new you want. And I think here, the learning is actually, this is you don't want to just QA your data at the moment you set it up internally, which would be the same as the beginning, but actually regularly, like it's something that needs to, I believe should be done more regularly than it is for most people.

Lior (10:47):

On a daily basis, basically.

Thomas (10:49):

Yeah. There's a lot of crap that you discover like edge cases and outliers and things that broke. And you didn't know that I don't know, three different names and reported different things,

Lior (11:06):

Missing, installs,

Thomas (11:07):

All the fun stuff, the fun stuff. It's not getting any better on the side.

Lior (11:16):

So let me ask you, if you had the ability, which KPIs would you kill?

Thomas (11:21):

Uuyeah, I mean, one that is very useful, but I think at some point we'd be killed is what, what I call the tap, which is what I call the clique at the very top of the funnel. When you, when you show a nod to somebody, typically the journey would be that it taps on the ad reach app store. And then on the app store decide if instills enough, this is very much of an always you think about it. One because the more new products develop, especially like on stories and so on and on YouTube ads and like this big and mentoree, there's a lot of youth retribution that happens, which actually is going to be very fun next year is Apple new system. Yeah. What I'm seeing killing the clicks is because the young audience more and more, they just go and do by themselves. So you can see the influencer even on a bunch of channels. And the reason I'm saying that is also like I see a lot of people judging the Facebook ads based on like the quality of the creative based on CTR. Almost recommend always to recommend, to look at IPM, which is basically concatenating, pre great an install rate together and going from the impressions straight to the install. I don't think the click is extremely,uvaluable in the economics because at the end, I mean the platform, the networks that just want to transform what they have, which is diligence attention into, into money. And whether people click a little bit more and install a little bit less or the other way around, it doesn't matter to anyone in this, in this economic, nothing to do. The click is actually a sometimes misleading. That's at the very top of the funnel. Maybe the other one I would kill is sessions. Like I hate looking at sessions because they're, they're meaningless. I mean, you, you want to raise your session, change your icon. People don't recognize it because they don't know what this is to stop it. Like the reason I'm saying the session path is in one particular company, we noticed that we had like not massive, but a significant amount of session, which were doing absolutely zero activities. You even zero, anything. We're like what? I don't know what these people are doing and what's wrong with us, but we shouldn't factor them as something positive. And back then we defined two sets of events. One that we call engagement and the other one was called relay. Let's say there was low engagement actions and high engagement actions. And actually the sessions, there were not even in the low engagement action. They went nowhere because we were like, if somebody opens the app and does nothing at all, should we really count that? Like as attention as a positive signal, I think not the session that doesn't do anything, they need at least a little bit or do something. So I can session.

Lior (14:21):

Click on something, make some action.

Thomas (14:23):

Exactly. Which see, I went to very fundamental metrics and I wouldn't call them. I went to very, very basic metrics. Like those are not complex and very deep stuff there. They actually, the first layer is the least important.

New Speaker (14:45):

I completely with you on that. So we are, we are looking at the same thing usually. And we saying like, sessions don't make sense and clicks on make sense, unless there is some user actions behind it. Otherwise, what does it's worth X amount of clicks or X amount of sessions.

New Speaker (15:03):

Exactly.

Lior (15:04):

And then if we looking at KPIs, which KPIs will you encourage people to start using? Because you don't think they're using them enough. Or, you will love to see them more as common thing.

Thomas (15:15):

Because of my background, I'll give you two that I think are very important. One for I'll give one for early stage and later stage again, like I did before. Like for earlier stage, a lot of people they come to, I want to scale acquisition a bit, or I want to even start acquisition from zero. Honestly, many times the first KPI I'm looking at is what kind of role new bird install or user or download they're seeing. Like you define it. And the reason for that is if that number is low, your app can work organically and with community and featuring and so on. But there's no way you're going to make Facebook profitable. If you're making 50 cents out of, out of a user in the US that it's never going to work miracles here, it's sort of a, this one is more of a one that I want, I want to understand, like, is this going to be maybe possible? Or I'm already wasting my time and it will, if it's very low. So that's the first one, like for the early stage or later stage, I see a lot of marketers looking at, I mean, in gaming, the, they look at this very often, but in, in, non-gaming not so much like a lot of people looking at the return on ATV or even predicted LTV, like over like years. UI think the payback period is something that,uanyone doesn't, I mean, you don't need to use it all the time, but at least run checks on that. Like, Hey, LTV, looks hot. Like actually, when, when is the payback mom, do I take six months to become profitable? Or does it take one month? Does it take a year? I think people underestimate the uncertainty behind typically like this year in spring, when the outbreak started, it was sorta people that were like, Oh, actually there's a lot of inventory that is cheap. And yeah, I'm in Q2 was great for many verticals in app that were not related to the physical world because retailers went out, because a lot of things went out and, and they were showing me amazing numbers. And I was like, wait a moment. Like, so you're telling me that people are going to keep covering forever at the same rate. And I was like, imagine destroys. This is really massive in terms of purchase power. For a lot of, a lot of people are out of jobs and, and I've just less income available in the States is not going to pay them forever. You're telling me that in a year from now, your assumption is that people are going to keep spending on your entertainment, secondary thingy. I mean, this is not like life necessity as like, I think you're making assumptions here for the future that are dangerous because if this doesn't happen and suddenly those future conversions are lower, then maybe this money you're spending now is actually unprofitable and you don't even know it. And this is where, I mean, it's good to project for later and it helps you a, but what I'm trying to say is it's good to understand how high, like basically the longer the payback window is the more risk you take, that things may change in the environment and that you may have been wrong. Yeah. So that's something.

Lior (18:36):

I completely agree. I mean,

Thomas (18:38):

Then companies would put like, okay, we'll accept longer payback period, because we know eventually we'll make more money. I mean, there is, there is different like levels of aggressivity you can put with that. And what you put the cursor is very personal. But my recommendation here is if you don't know, what is your payback by? You need to know, do you need to wait for the monthly renewal or after the third year already? Like let's at the a hundred. And, and for me, like the moment you reached the spot that you repaid, the marketing cost is very important to be, to be close enough. And you think that's more than 12 months for me is ridiculous. Like ridiculous. It shouldn't be done,

Lior (19:19):

Especially nowadays. I think that actually we haven't seen you at the edge of this crisis. I think it just now starting to roll out to a lot of companies.

Thomas (19:29):

Yeah. Hopefully I'm wrong. But I also think like, I hope the direct outcome is actually closer to the end and the beginning, but I think there are very indirect outcome that are hard. I mean, governments are not going to be able to do sustain, like the kind of spending level that they're doing for months. And that it's just, I don't know, like I'm not an economist and I don't see the future for society. It's just, it's high and set that time, which is clear, be careful about in business, making your prediction over very long time. Because if anything, this has proven that we have no clue about what's coming to us and all the black ones we might encounter on the way.

Lior (20:10):

Like Apple privacy policy that basically going to scrap out attribution almost for most companies.

Thomas (20:17):

Well, I wouldn't call that one, a black one because we saw it coming from far away. We just decided to ignore it. So that, that one was not a Blackfoot was coming like four years. And it's still not there again.

Lior (20:34):

Postpone it again for another six months. So it's somewhere around June.

Thomas (20:38):

I heard about, but I heard, I hear, I hear a lot of things.

Lior (20:42):

So it will be interesting to see the, now this one going to actually affect the industry in general. Do you see today, people actually starting to change the way that they are tracking data or using data due to these changes.

Thomas (20:56):

Yeah, but it's more about preparing to change then already changing because it's like, it's not the switch, but at least like maybe not so much in terms of in executive session, like, Oh, we already switched the mindset and how we look at data, but rather, no, sorry, not the center, like how they actually construct the data today, but rather already about evolving mindsets about ha what did we, what would it look like if we remove impression level data, what would it look like if we actually don't have any conversion after the first day? So more like thinking hypothetically like this, one of these questions that don't necessarily end up in data that is very actionable for marketers,

Thomas (21:37):

But I think at least it's good to have like this reflection around. Yeah. What would my data look if this, and if that can be done for other things, not in things that Apple have said, but now it's quite interesting.

Lior (21:51):

Do you see any effect on it on the way that we're doing campaigns? Are we doing, we setting up marketing nowadays?

Thomas (21:56):

Well, currently not really. One big reason is that non of Facebook, Google, Apple snap, or Twitter, or any signs or Pinterest, I've actually given any possibility to test new attribution. We can run a few tests on, on some networks. I'm not self attributed, but I think as soon as the, as long as the self attributing network are not moving, it's not really the same, you know, they're, they're really moving the industry.

Thomas (22:27):

So, no, there's not a lot of happening right now. If anything, I, I published the thoughts bill saying that one thing that is happening is there's more people without idea for right now. And those you can't catch them on network at the moment. Twitter and snap are running pilots on this, like in the very short term, what it means is yeah, maybe it's a, it's a good time to rethink your marketing. I'm not saying you should shift money right now, and I'm not, I have no to where you should shift it. But I think because we're before the storm, it's, as if we're in the eye of the, of the cycle, how do you call it? Like the quiet moments and that's where we are basically like the water's clear, but we just, we just know it's coming. I think it's a good moment to think about this topics about a hOW WE DO IT. Oh, I we're doing lastly, we're doing first is still less than still. How do we, how do we understand the interaction between different channels and what is even our media mix and how to think about it? I think it's a good reflection moment. Exactly. At the moment, some people are panicking and jumping into a boat that can actually be actioned yet. I think it's a good time to sit and relax and think about the big picture,

Lior (23:54):

You know, because we now talking more and more, and I think that also wrote an article about it to move to a first party tracking rather than having the old method of having relying on third parties all the time. And I think that this is one of, at least my trends for 2021 that companies need to move more and more into first party attribution and first party tracking at least as much as possible and rely less and less on third parties. And this is something that's going to be quite interesting to see after Apple will actually announce what they're doing, because right now I think they gave us enough details, but it didn't give any details regarding what's going to happen with Facebook or Google, as you said, it just unclear.

Thomas (24:38):

Yeap. Let's see.

Lior (24:44):

You're going to have a lot of fun. I,

Thomas (24:46):

I kind of agree with you, like sort of, of course, like less, less third party first party, eh, maybe that would be a very long discussion for this podcast, but that would be interesting to see how you, how you see first battery first party attribution panel, because that sounds like typically the one thing to me that is never unfairly first party, I mean, you interact with another and the platforms in the middle, I sit for other things and it's true that this is even bigger than the app. If you look outside, I mean, Firefox and Safari browsers already killed like third party cookies for a little bit and Google will do it when they have their, the privacy sandbox, which is something that I think is fundamentally wrong here. But at the end of the day, all these trends, whether it's the, IDFA the third party cookie different legislation that are around it, it's all pushing for first party data. We're constructing your audience is the most important. And.

Lior (25:51):

Exactly.

Lior (25:52):

What one, I think it's good because it's less reselling and so on, but it's got a consequence that we have to check, which is favoring concentration. And not all of them. I'm not talking on leads, Facebook and Google are going to be even better off than they already are. And I bet they will. But at every level you see it in, in gaming companies where a lot of the M&A market today is to acquire audience do more. And basically they're not acquiring games. They're acquiring audience, they're acquiring a user acquisition channel, like instead of, you know, putting a hundred million on Facebook ads, you put a hundred million in buying that studio that has, I don't know how many million users it's just user acquisition as well. And it creates the consolidation trend, I think at many levels at the network level, at the app level. Um yeah, so it's not necessarily all good. I like to have a diversity of apps. Yeap. First body is critically important. And you see it in, in the media industry. I mean, they're really trying to bring back what they've externalized first party data.

Lior (27:06):

I think that we're going to see change. I think we're going to see a huge change with this first party data and also with attribution. So at the end of the day, Apple will still need to give you an alternative, right? That you can run it yourself and Facebook and Google as well, based on the consent of the user. And.

Thomas (27:23):

I don't know,

Lior (27:23):

I think, I think that this is a development that we're going to see coming soon. And this requires that you be able to track your data and actually store it locally. But yeah, it's a, it's an interesting challenge of, I would love to, to hear how other people looking at it and how they can solve it.

Thomas (27:44):

A man, this is a hard problem don't have a solution. And there's lots of things we don't know yet. I mean, there isn't this coming changes so big is because Apple have a very different perception of whatever attribution is should work to the mechanics that we've used until now. It's hot, we're hot somewhere in between next year. I'm actually very interested to have this deeper question about like, Oh, how do I adjust? How do I block my device id to scare, but more, how does that actually work and what you can already see? Like there are a few products moving a lot more on measuring incrementality and paid organic ratio and the fate of correlation that has always interested me quite a bit and becoming even more important. But at the end of the day, whichever attribution method you use, like it's good to be informed by a couple of different models to just validate if you're seeing it right or wrong.

Lior (28:52):

I agree. I completely agree about that. So Thomas, we rubbing to the end of the podcast. Any, anything you would like to add before we say goodbye?

Thomas (29:02):

Uno, it was,uyeah, it was nice. Very happy to it today as a cool, a cool discussion. I leave the last word to this, that I said that it's not time to panic. It's time to sit down and think bigger. So I invite you all to think about that.

Lior (29:19):

So thank you very much for joining. It's been really a pleasure. I think it's a great conversation, a lot of stuff to take out of it. And I see you soon. Bye-Bye

Speaker 1 (29:40):

Thank you for listening to the, what the data podcast.