Mysql and node async.waterfall

I am trying to find out if the record exists in my database table. If it does not exist there, then I want to add it to my database. I use node and using async.waterfall. But something is wrong with my code and I could not find what. It does not add the record to my database. Any suggestions?

var mysql = require('mysql'); var connection = mysql.createConnection({ host : 'localhost', user : 'user', password : 'password', database : 'database' }); for (var m = 0; m <= (urls.length-1); m++) { var Myurl = urls[m]; var ThisName= TheNames[m]; //IIFE function. I omitted the error handling for clarity. (function(Myurl,ThisName){ async.waterfall([ //First find out if the record exists in mydata function(next){ connection.query('SELECT * FROM mydata WHERE UrlLink=? LIMIT 1',[Myurl],next) }, //If the record does not exist, put it in mydata function(results,next){ if (results.length==false){ console.log("New Thing!"); //Do some stuff here, request(Myurl...) to find TableName. var post = {UniqueUrl:Myurl,ThingName:ThisName,TheName:TableName}; connection.query('Insert INTO mydata Set ?', post,next); }; }; ], //Final callback function(err, results) { connection.end(); }; ); })(Myurl,ThisName); };

EDIT:

I got an error with the suggested answer TypeError: object is not a function, at the line return next() in the if statement. In my real code I actually do something like that (when I use the suggested answer):

async.series([ //First function function(callback){ //Some calculations... callback(); }, //Next function function(callback){ var i = 0; //Loop async.whilst( function() { return i <= thefooz.length-1; }, //The innerCallback: function(innerCallback){ //Some calculations where i get urls and TheNames. async.forEachOf(urls, function(Myurl, m, eachDone) { var ThisName = TheNames[m]; async.waterfall([ function(next) { connection.query( 'SELECT * FROM mydata WHERE UrlLink=? LIMIT 1', [ Myurl ], next ); }, function(results, next) { if (results.length !== 0) { return next(); } console.log("New Thing!"); //Do some stuff here, request(Myurl...) to find TableName. var post = { UniqueUrl : Myurl, ThingName : ThisName, TheName : TableName }; connection.query('Insert INTO mydata Set ?', post, next); } ], eachDone); }, function(err) { if (err) throw err; // or however you like to handle errors connection.end(); }); setTimeout(function() { i++; innerCallback(); }, 10000); }); //close my calculations in innerCallback. }, //close innerCallback. ); //close asyns.whilst callback; } //close function(callback) ], function(error){ if (error) return next(error); });

I am really stuck in callback hell...

-------------Problems Reply------------

An (untested) version that should do the same, but with some bugfixes and enhancements in terms of leveraging more of async's goodness:

var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'localhost',
user : 'user',
password : 'password',
database : 'database'
});

// Process all url's in parallel (at the same time). If you to process them
// sequentially (one after another), you can use `async.forEachOfSeries()`
// instead. Or, if you want to limit the number of parallel requests, use
// `async.forEachOfLimit()`.

async.forEachOf(urls, function(Myurl, m, eachDone) {
var ThisName = TheNames[m];

async.waterfall([
function(next) {
connection.query(
'SELECT * FROM mydata WHERE UrlLink=? LIMIT 1',
[ Myurl ],
next
);
},
function(results, fields, next) {
if (results.length !== 0) {
return next();
}
console.log("New Thing!");
//Do some stuff here, request(Myurl...) to find TableName.
var post = {
UniqueUrl : Myurl,
ThingName : ThisName,
TheName : TableName
};
connection.query('Insert INTO mydata Set ?', post, next);
}
], eachDone);

}, function(err) {
if (err) throw err; // or however you like to handle errors
connection.end();
});

Instead of using forEach*, you can also use each*, but since you need the array index as well forEach* is more convenient.

Relevant documentation:

  • async.forEachOf
  • async.forEachOfSeries
  • async.forEachOfLimit
Category:javascript Views:0 Time:2018-09-13

Related post

  • The "err" argument when using Async waterfall in node.js 2011-01-06

    I am trying to execute a series of functions, each passing the callback to the next. Right now it looks like this (excuse any minor errors, I am rewriting it as I post!): function func1(callback) { callback(null, "stuff"); } function func2(input, cal

  • What is the difference between async.waterfall and async.series 2012-02-13

    The nodejs async module: https://github.com/caolan/async provides 2 similar methods, async.waterfall and async.series. What is the difference between them? --------------Solutions------------- It appears that async.waterfall allows each function to p

  • How do I call an external function from within a chain in async.waterfall in node.js 2017-06-22

    I have a problem when trying to use modbus-tcp in async.waterfall Basically I have to complete one modbus read before continuing the other. async.waterfall([ function startInfo(next) { console.log("Dummy Starting A") next(null, 'A') }, function

  • async.waterfall jumping to end before functions complete 2014-05-15

    Hope you guys are well? I am diving into Node.js and having to relearn a lot of the ways in which I used to code - and having to retrain myself to asynchronous ways... I am writing a server-side script (like a java POJO), that will run either from th

  • How to setup Replication or Federation between MySQL and Derby? 2010-03-10

    Perhaps this is not possible yet, but... What's the easiest way to setup Replication or Federation between MySQL and Derby? I need a way for data updates/inserts/deletes into a certain table in Derby to automatically flow into an equivalent identical

  • Is it a good idea to use MySQL and Neo4j together? 2010-03-29

    I will make an application with a lot of similar items (millions), and I would like to store them in a MySQL database, because I would like to do a lot of statistics and search on specific values for specific columns. But at the same time, I will sto

  • How can I use MySQL with Node.js? 2011-10-19

    I've looked around for a bit, but I haven't found a way to use Node.js to access MySQL. How would I do that without using a side program? --------------Solutions------------- The way to go is to download node.js, install mysql server (I had it bundle

  • Is my conceptual misunderstanding of node async wrong? 2011-10-23

    I'm playing around with Node.js as a driver for some build time javascript optimization. I've the following Jake file: fs = require("fs") uglify = require("uglify-js") desc "Uglify JS" bundles = edit: [ "jquery",.... ] login: [ ... ] directory: [ ...

  • Redis and Node.js and Socket.io Questions 2012-01-20

    I have been just learning redis and node.js There are two questions I have for which I couldn't find any satisfying answer. My first question is about reusing redis clients within the node.js. I have found this question and answer: How to reuse redis

  • How can I query MySQL with node.js? 2012-03-29

    I am going to make a client-server application using Node.js, and I want to use a MySQL database for storing some of the data. I need to know how to query a database with server-side JavaScript. --------------Solutions------------- Search the module

  • MongoDB, Mysql and relationships 2012-04-23

    I'm creating an online chat. Context (if needed): So far I was using PHP/MySQL and AJAX to do the job but this is not a healthy solution as I'm stuck with a "pull" type application with concerns about scalability. I read about the "push" method alter

  • Differences Between MySql and MS SQL 2008-08-14

    I'm an ASP.NET developer who has used Microsoft SQL Server for all my database needs (both at work and for personal projects). I am considering trying out the LAMP stack for some of my personal projects. What are some of the main differences between

  • Stored Procedures, MySQL and PHP 2008-09-23

    The question is a fairly open one. I've been using Stored Procs with MS SQLServer for some time with classic ASP and ASP.net and love them, lots. I have a small hobby project I'm working on and for various reasons have gone the LAMP route. Any hints/

  • What do you need to take into consideration when deciding between MySQL and Amazon's SimpleDB for a RoR app? 2008-10-02

    I am just beginning to do research into the feasibility of using Amazon's SimpleDB service as the datastore for RoR application I am planning to build. We will be using EC2 for the web server, and had planned to also use EC2 for the MySQL servers. Bu

  • Problem with conflict between mysql and math.h 2008-10-07

    The problem is that the compiler says that there is a redefinition of a function between a library that belongs to MySQL and math.h from the std library. I have been over this for two days and I still can't figure it out. Has this ever happened to an

  • How do you write a case insensitive query for both MySQL and Postgres? 2008-10-15

    I'm running a MySQL database locally for development, but deploying to Heroku which uses Postgres. Heroku handles almost everything, but my case-insensitive Like statements become case sensitive. I could use iLike statements, but my local MySQL datab

  • MySQL and SQL Server for same application 2008-11-06

    What are good ways of achieving this DB agnosticism without actually coding two DAL's? I've heard that the Data Access Application Block is suitable for this. --------------Solutions------------- You could also look at the various ORM libraries like

  • How do I install MySQL and the Python MySQL package on OS X Leopard? Or how do I learn about being a web developer using OS X? 2008-12-27

    I'm new to the Mac OS X, and I'm just about ready to throw my brand new MacBook Pro out the window. Every tutorial on setting up a Django development environment on Mac OS X Leopard is insidiously wrong. They are all skipping over one step, or assumi

  • How can I get started with Apache, MySQL, and (PHP, Perl, Python) on a Mac? 2009-01-12

    What do you recommend for setting the MAMP development stack? I've seen the quickstart thing at http://www.mamp.info, but I don't really like that approach. I'd rather have a full-fledged Apache that I can install any number of modules into. I rememb

  • Difference between mysql and sql server? Performance, features,-? 2009-03-20

    What are the differences between MySQL and Sql server? What are the parameter to chose between the two? --------------Solutions------------- Several questions here related to DB comparisons of various flavors: http://stackoverflow.com/questions/48187

  • MySQL AND Filemaker Pro? 2009-03-24

    I have a client that wants to use Filemaker for a few things in their office, and may have me building a web app. The last time I used, or thought about, or even heard of, Filemaker was about 10 years ago, and I seem to remember that I don't want to

  • MySQL and Check Constraints 2009-04-01

    I have inherited an application that uses MySQL and that is used by a PHP front end. The guy that wrote this system has gone to some fairly convoluted lengths to ensure that codes that users enter are valid - and tat means that these codes also exist

  • MySQL and SQLite differences in SQL 2009-04-15

    I'm writing java application that is using both SQLite and MySQL using JDBC. Are there any differences in SQL for those databases? Can I use same queries for both SQLite and MySQL, or is there any db specific stuff, that doesn't work on the other one

  • Rails, MySQL and Snow Leopard 2009-06-13

    I upgraded to Snow Leopard using the disc we got at WWDC. Trying to run some of my rails apps now complains about sql (in /Users/coneybeare/Projects/Ambiance/ambiance-server) !!! The bundled mysql.rb driver has been removed from Rails 2.2. Please ins

  • Randomizing pages in Wikipedia with MySQL and Perl? 2009-07-04

    I found a perl script that manages randomizing the wikipedia articles in Wikipedia here. The code seems to be slightly computer generated. Due to my present interest in MySQL, I thought you could possibly have the links and related data in a database

Copyright (C) dskims.com, All Rights Reserved.

processed in 0.225 (s). 11 q(s)