Processing Large Data Volumes using PK Chunking & Hyperbatch with Daniel Peter

Peter Coffee (@PeterCoffee) is a repeat guest on the TechnologyFlows podcast.  With a seemingly bottomless bag of statistics, quotes, anecdotes, and historical references to back up his opinions it always makes for a fascinating discussion. This conversation was recorded live in front of an audience at Dreamforce 2017 in San Francisco.  Peter's energy and enthusiasm were in full effect even as we were setting up the podcast recorder!   I hope that you enjoy the show.  Please leave feedback on the blog at TechnologyFlows.com or tweet me directly, I am @matmorris Recorded in November 2017 This podcast interview was first published by Technologyflows.com © TechnologyFlows

In this episode I will be speaking with Doug Ayers (@DouglasCAyers) about his approach to interpreting requirements and evaluating the options to architect a solution. As an experienced web application developer coming to the Salesforce platform more than 5 years ago, Doug already had extensive technical skills which he could to use to grasp Apex and Visualforce, but how did he adapt to working on a platform?  I asked Doug about his own experiences and the lessons which he now teaches to fellow developers.  We discuss the value of asking more questions to uncover the depths of requirements and how the significance of considering the way a  solution will be maintained over its lifetime is as important as how easily it can be created. Doug's a problem solver, as can be seen from his GitHub repository which hosts multiple utilities he has created to fill gaps in Salesforce functionality.  For example when Doug saw a question on Twitter asking how to migrate Salesforce notes & attachments data into the new Notes feature, he did some research and found that the feature wasn't available out of the box.  So he wrote it himself!  He did the same for migrating file attachments to Chatter files, he created an automation Bot for Chatter groups, enhanced Campaign member management, the list goes on.   You can read for yourself on Doug's blog at douglascayers.com. You can find Doug on Twitter @DouglasCAyers. Please leave feedback on the blog at TechnologyFlows.com or tweet me directly, I am @matmorris Recorded in June 2017 This podcast interview was first published by Technologyflows.com © TechnologyFlows

In this episode I will be speaking with Daniel Peter (@danieljpeter) about processing large volumes of data on Salesforce. Daniel is Lead Application Developer at Kenandy, an ISV who had built an ERP solution on the Salesforce Platform. Daniel’s first hand experience of how the Salesforce multi-tenant database behaves has lead him to develop techniques for processing tens of millions of records. He will describe the techniques which he has refined to ensure SOQL queries are executed with consistent reliability and not fall foul of the most common exceptions relating to row selection, which are: Non-selective query Too many query rows returned Query time out during execution Daniel will explain how the Batch Apex query locator can be used to implement a technique called PK chunking which allows fine-grained control of the number of rows to be processed in each batch which largely overcomes the 3 common exceptions. Daniel has even gone as far as experimenting with parallel execution through his Hyperbatch open source project which you can download from GitHub. Whether your Salesforce database contains tens of thousands or rows or or if you’re up into the 10 of millions Daniel’s tips on working with multi-tenancy are a real eye opener as to what is possible when you design for scale from the outset. Please enjoy! Please leave feedback on the blog at TechnologyFlows.com or tweet me directly, I am @matmorris Recorded in June 2017 This podcast interview was first published by Technologyflows.com © TechnologyFlows

In this series of interviews I will be speaking with  people on the frontline of Salesforce architecture & development. In this first episode of the Architect Series my guest is Francis Pindar which is very fitting as he was one of the pioneers of the UK Salesforce Community scene and one of the first people that I came into contact with when I got involved with Salesforce back in 2010. Since then he’s continues to be a source of ideas and wisdom to me and countless others. Francis is: 5x Salesforce MVP (2012 – Present) Salesforce Certified Sales Consultant, Service Consultant, Developer, Administrator & Advanced Administrator. AWS Certified Architect – Associate TOGAF9 Certified Enterprise Architect BCS Certified Enterprise Architect Francis has worked in internet technologies for 20 years, the last 10 of which have been on the Salesforce Platform. He is a specialist in the areas of Enterprise Architecture Design, Business Process Improvement, Project Leadership, Business Analysis & Migration strategies.  Follow Francis: https://twitter.com/radnip http://www.radnip.com/ https://www.youtube.com/channel/UCbPyfToAJcUCDxIEuqlfVqA Recorded in June 2017 This podcast interview was first published by Technologyflows.com © TechnologyFlows

In conversation with Peter Coffee (@petercoffee) at the November 2016 French Touch Dreamin' conference in Paris. Peter is VP for Strategic Research at Salesforce and his Twitter bio describes him thus; "I work with app developers and CIOs to build a global community of innovation, service and success". As you will hear, Peter is never stuck for a quote, a metric or a research finding to support his arguments that when building modern software architectures, there is no advantage in duplicating what has been done in times past. Please leave feedback on the blog at TechnologyFlows.com or tweet me directly, I am @matmorris. Enjoy the show...  

This week we are paying a visit to the Dreamforce IoT Zone. We’ll hear from Doug Merrett and his robot Sparky.  Doug is a Principal Enterprise Architect at Salesforce. Even though it is 2015 and these techniques have been used for decades to build cars and sort parcels for delivery, not everyone makes the connection that the software which they use on their phone or web browser can affect and interact with the physical world, and for me this is really what the IoT Zone at Dreamforce is all about, helping people imagine what is possible and apply what they have learned to their own situation. The technology of IoT has progressed rapidly and has made its way into many products we come into contact with every day. Before we get to Doug and Sparky, we have a tour of the IoT Zone from Pat Patterson.  Pat is Developer Evangelist Architect at Salesforce https://www.linkedin.com/in/metadaddy and he walked and talked me around some of the major products on show which are connected through the new Salesforce IoT cloud.  You can tweet Pat @metadaddy All the technology related zones at Dreamforce on the second floor of the Moscone West building were buzzing and they clearly made an impact on Salesforce CEO Marc Benioff as he gave mention in his Q&A session to both Adam Seligman, Senior VP Developer Relations and to Pat. I really hope that you enjoy this weeks show.  You can subscribe to Technology Flows on iTunes and please feel free to leave feedback on the blog at TechnologyFlows.com or tweet me directly, I am @matmorris

It’s been a month since Dreamforce 2015, so I thought that it might be interesting to replay some of the hot topics of conversation from around the Dev and Admin Zone. I was joined in the AdminZone podcast studio by two experienced and talented people from the community to help me understand what we need to make the transition to using Lightning. LeeAnne Templeman is a Developer Evangelist at Salesforce, she can be found on twitter @leeandroid. Justin Edelstein is co-founder and CIO at, Arkus a Salesforce partner based in New York, he’s a Salesforce MVP and you can tweet him @JustEdelstein. Please feel free to leave feedback on the blog at TechnologyFlows.com or tweet me directly, I am @matmorris Enjoy the show!

Hello and welcome to another episode of the Technology Flows podcast. My name is Matthew Morris and I’m a Salesforce admin and MVP. Each week I’ll be brining you the voices and opinions from the world of Salesforce. This week I have for you a conversation with Peter Coffee who is the Vice President for Strategic Research at Salesforce. Many of you will recognise Peter as the host of pre-keynote conversations at many Salesforce events.  I think I am right in saying that it is his voice which announces every keynote session at Dreamforce, which he has to pre-record in the run up to the conference. If you’re interested in “the future”, then Peter is you man.  He leads the emerging trends track at Dreamforce, which from what I have seen gives a 2-3 year view into the future of Salesforce and the technology industry. Before he was hired by Marc Benioff in 2007, Peter was a technology journalist.  He’s also a graduate of the Massachusetts Institute of Technology and back in the day he even wrote a book entitled How to program Java. If you want to improve your understanding of how technology and business work together and where they are leading us Peter is your man. I sat down with Peter in the press room at Dreamforce on the very last day with the hope that he could help straighten out my thoughts about all the announcements and technologies which were on show that week.  Clearly my head was still spinning as you’ll hear we made reference to the I’m a Mac adverts which ran from 2006 to about 2009, so a good 6 years ago.  Not the 2 or 3 that came out of my mouth.  Things move fast today, it seems. You can find Peter on twitter @petercoffee Please feel free to leave feedback on the blog at TechnologyFlows.com or tweet me directly, I am @matmorris. Enjoy the show...

This week I’m speaking with Don Robins, who is an award winning instructor for Salesforce University and he has been a pioneer in providing education and advice for Salesforce admins and developers wanting to learn how to exploit Salesforce’s new Lightning technologies. Don and I got together in a San Francisco bar, directly across the road from the Dreamforce conference which had just closed. This conversation came almost exactly a year after our first conversation about the release of Salesforce Lightning components. And we started by talking about how quickly the time had passed, and how people (including me) can catch up and start understanding how to exploit the power of Lightning. Over the past 18 months, Don has produced some excellent materials which introduce developers and admins to building with Lightning. Top of the list is Don’s Salesforce University online course which you can access for FREE. Don also produced a series of 6 blog posts which introduce the concepts of lightning and are a great primer for the training course. Lightning is really here, and I really recommend that anyone associated with the Salesforce eco system read these blog posts.  If your job is as Salesforce developer or Admin then you should to continue to the online training, which is a high quality Salesforce University production with video and slides. All of these great resources can be seen by going to the link http://bit.ly/lightningDon. I hope you enjoy the show, please feel free to leave feedback on the blog at TechnologyFlows.com or tweet me directly, I am @ M A T MORRIS

This time we hear from 2 people right at the heart of Salesforce’s Lighting technology evolution. Skip Sauls is a Director of Product Management at Salesforce and he is working alongside Doug Chasman who is Principal Architect on the Platform team.  The work of their team is making it possible for developers and admins to customise Salesforces’ user interface like never before. Doug was one of the two original architects of Visualforce, and I started by asking him if Lighting components will replace VisualForce. This interview was recorded about 2 weeks prior to the announcement of the new Salesforce Lighting Experience, so we we’re able to discuss that on tape.  Skip and Doug were extremely open about the technologies which they have waiting in the wings and Lighting Out sounds like it has the potential to be a transformational technology. Dreamforce 2015 is taking place next week from the 15th September, if you’re going be sure to seek out Skip and Doug.  If you’re staying at home, watch the online broadcasts and keep an eye on the Salesforce YouTube channel for the sessions to be published in the weeks following the conference. Take a look at the blog at http://www.technologyflows.com, you can tweet me @matmorris.     Thanks for listening!