How to build batches/buckets with linq

I need to create batches from a lazy enumerable with following requirements:

  • Memory friendly: items must be lazy loaded even within each batch (IEnumerable<IEnumerable<T>>, excludes solution building arrays)
  • the solution must not enumerate twice the input (excludes solutions with Skip() and Take())
  • the solution must not iterate through the entire input if not required (exclude solutions with GroupBy)


The question is similar but more restrictive to followings:

  • How to loop through IEnumerable in batches
  • Create batches in linq

-------------Problems Reply------------

Originally posted by @Nick_Whaley in Create batches in linq, but not the best response as the question was formulated differently:

Try this:

public static IEnumerable<IEnumerable<T>> Bucketize<T>(this IEnumerable<T> items, int bucketSize)
{
var enumerator = items.GetEnumerator();
while (enumerator.MoveNext())
yield return GetNextBucket(enumerator, bucketSize);
}

private static IEnumerable<T> GetNextBucket<T>(IEnumerator<T> enumerator, int maxItems)
{
int count = 0;
do
{
yield return enumerator.Current;

count++;
if (count == maxItems)
yield break;

} while (enumerator.MoveNext());
}

The trick is to pass the old-fashion enumerator between inner and outer enumeration, to enable continuation between two batches.

Category:c# Views:0 Time:2017-12-19

Related post

  • How to build batch processes with rails 2010-06-04

    I'm using ruby on rails 2.3.8 and I'm kind of knew at this technology. I'm using Windows 7, but I'm planning to hire a Linux hosting soon. I'd like to know how to build processes that runs every night for things such as: checking stuff in the db, upd

  • How-to build AutoComplete / Suggestions with Lucene.NET? 2010-02-17

    How-to build AutoComplete / Suggestions with Lucene.NET ? --------------Solutions------------- Very simple response here, it'll get you started. The SpellChecker project, which is part of lucene.net contrib, will give you Google's "Did you mean?" lik

  • Good tutorial on how to build a wiki with PHP & Mysql? 2010-10-10

    I'm searching for a good tutorial on how to build a wiki with PHP & MySQL. I need something that will show me how to structure the mysql database and which algorithms to implement for the text diff etc. The only one that I've found so far is this

  • How to select single record with Linq? 2011-06-16

    i have this table **ID CityCode Price** 1 ABB 100 2 CAB 160 3 CAB 340 4 TRE 800 5 ABB 770 6 TRE 900 7 ABB 500 I want write down a query(linq) which will return the following result as show the code below CityCode ABB CAB TRE i want select a list with

  • How to simulate Derived table with linq to Entities 2012-01-03

    How I can simulate Derived Table with linq? Consider this Code: SELECT * FROM (SELECT * FROM Mytbl) AS tmp WHERE tmp.ID=1 How I can write this with Linq ? thanks EDIT 1: How to convert this to linq?: select convert( decimal(10,1), ROUND(A.c2*100,8)),

  • How to build an XDocument with a foreach and LINQ? 2009-06-16

    I can use XDocument to build the following file which works fine: XDocument xdoc = new XDocument ( new XDeclaration("1.0", "utf-8", null), new XElement(_pluralCamelNotation, new XElement(_singularCamelNotation, new XElement("id", "1"), new XElement("

  • How to make POCO work with Linq-to-SQL with complex relationships in DDD 2011-04-08

    I am struggling to find a way to make POCOs work with Linq-to-Sql when my domain model is not table-driven - meaning that my domain objects do not match-up with the database schema. For example, in my domain layer I have an Appointment object which h

  • How do I do this with LINQ? 2009-11-26

    I'm not entirely sure if I'm trying to do something a little too complex for this but what I essentially want to do is turn this: declare @callid int set @callid = 57 declare @update nvarchar(max) declare update_cursor cursor for select UpdateIdentit

  • How to use SQL 'LIKE' with LINQ to Entities? 2010-06-22

    I have a textbox that allows a user to specify a search string, including wild cards, for example: Joh* *Johnson *mit* *ack*on Before using LINQ to Entities, I had a stored procedure which took that string as parameter and did: SELECT * FROM Table WH

  • How to build HTML Documentation with Sandcastle 2010-09-03

    How to build CHM / HTML Documentation with Sandcastle? Can build HTML file from XML or have to used .cs and dll files? --------------Solutions------------- How to build CHM / HTML Documentation with Sandcastle? Use the Sandcastle Help File Builder. C

  • How to build libcurl.so with a different target name? 2010-11-02

    I am using libcurl for my utility and its working very well till now for all Linux platforms. I downloaded, unzipped and simply followed the instructions given without any changes. My product uses the libcurl.so file and is linked dynamically. The .s

  • How to build a website with Php that collects articles? 2010-12-29

    I have a quick question. I'm trying to build a website with php that collects articles from different blogs. How would I code this in php? Would I need some type of regex statement? All I need to do is grab the articles from the specific pages. An ex

  • How to build boost library with open64 compiler? 2011-04-27

    I would like to give AMD's open64 compiler a try, but for this I first need to build some boost libraries (current version 1.46_1) using this compiler. Unfortunately, open64 is not among the toolsets provided by the boost.build system, see e.g. here

  • How to flatten nested objects with linq expression 2011-06-21

    I am trying to flatten nested objects like this: public class Book { public string Name { get; set; } public IList<Chapter> Chapters { get; set; } } public class Chapter { public string Name { get; set; } public IList<Page> Pages { get; s

  • Batch delete with Linq to SQl 2011-09-30

    Possible Duplicate: linq to sql batch delete I have a list of id's as List<int> ids. Now what I need is to delete the id's of the list from the database at a time, like we do using Where in (5,6,7,8) In SQL. What will be the LINQ statement for

  • How can I select data with linq to sql from another query? 2011-11-30

    How can I select data with such linq to sql code, it's something wrong, i must compare WHERE, from st1 and st2, but something wrong. Also spaller didn't light this variables in where var st1 = (from a in db.RouteDetail where a.Station == "Гродно" sel

  • UrlRewriting how to build correct redirect with sub folders 2012-03-01

    Hi I am a new one UrlRewriting, I have old urls like http://www.domaine.com/data.aspx http://www.domaine.com/data/page1.aspx http://www.domaine.com/data/page1/page2.aspx http://www.domaine.com/data/page2.aspx I changed data to new-data how to build U

  • How to build streaming API with Ruby Grape? 2014-06-26

    I want to build a API with Grape, which could be POST file through http streaming. How to do that? PS grape example of rack-stream couldn't work error info: HTTP/1.1 503 Service Unavailable --------------Solutions------------- I want to make grape ca

  • how to add sort direction with linq for sql? 2009-11-11

    Suppose I have linq expression q, then I want to add a sort to this query: q = q.OrderBy(p => p.Total); but for sort, there is desc/asc option for SQL, how to add it in above linq expression? --------------Solutions------------- OrderBy will sort

  • How to select static text with LINQ query? 2010-01-03

    I have a listbox and I want its ItemsSource to be all the items of an ObservableCollection + an additional static text such as "All Items." Right now I just have something similar to: listbox1.ItemsSource = from Car c in Cars select c.Model I know I

  • How to build a WS with CXF and Spring? 2010-10-19

    I'm trying to build a WS with CXF. I'm following this article http://www.ibm.com/developerworks/library/ws-pojo-springcxf/ But I have 2 questions: 1) http://stackoverflow.com/questions/3969268/which-maven2-artifacts-are-necessary-to-build-a-ws-with-c

  • How to enforce unique constraint with LINQ to SQL 2010-10-25

    I'd like to enforce a unique constraint on two columns with LINQ to SQL. I have got a constraint configured on a SQL table, but is there a way to use it with LINQ to SQL? Unique constraint is applied to two columns, both of which are foreign keys. Th

  • How to Split a string with LINQ and get the specified token 2010-11-02

    I have a tab-delimited string in the following format: string line = "D\t892270\t88418\t6\t2452927\t-99999\t-99999\t12"; What I want is to get the value of the token specified through indexToGet. I have written following LINQ query to do this which I

  • How to build boost libraries with bcc32 v6.31 + Borland XE? 2011-02-10

    I am trying to build boost_1_39 provided with Borland Rad Studio XE using WinXP. The latest version of Developer Studio provides boost_1_39 installer, but not the prebuilt libraries. Using bjam: \tools\jam\src\bin.ntx86\bjam --build-dir=c:\boostsourc

Copyright (C) dskims.com, All Rights Reserved.

processed in 0.090 (s). 11 q(s)